QUESTION ANSWERING

CharBERT: Character-aware Pre-trained Language Model (2020.coling-main)

Challenge:	Conversations are the natural communication format for people.
Approach:	This tutorial will survey the cutting-edge methods for summarizing written and spoken conversation.
Outcome:	This tutorial will examine the cutting-edge methods for summarizing written and spoken conversations, covering key sub-areas whose combination is needed for a successful solution.

Copied to clipboard

Wentao Ma, Yiming Cui, Chenglei Si, Ting Liu, Shijin Wang, Guoping Hu

jTLEX: a Java Library for TimeLine EXtraction (2023.eacl-demo)

Challenge:	Pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations . but these methods split a word into subword units and make it incomplete and fragile .
Approach:	They propose a character-aware pre-trained language model to tackle OOV problems . they construct contextual word embedding for each token from sequential character representations .
Outcome:	The proposed model improves on the existing models on multiple NLP benchmarks.

Copied to clipboard

Mustafa Ocal, Akul Singh, Jared Hummer, Antonela Radas, Mark Finlayson

SpanBERT: Improving Pre-training by Representing and Predicting Spans (2020.tacl-1)

Challenge:	Timeline EXtraction library provides Java implementation of TimeML annotations and tools for programmatic manipulation of Timeline graphs.
Approach:	jTLEX provides a Java implementation of TimeLine EXtraction algorithm and utilities for programmatic manipulation of TimeML graphs.
Outcome:	jTLEX provides a Java implementation of the TimeLine EXtraction algorithm, along with utilities for programmatic manipulation of TimeML graphs.

Copied to clipboard

Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, Omer Levy

A Benchmark Suite of Japanese Natural Questions (2024.starsem-1)

Challenge:	Pre-training methods like BERT mask individual words or subword units, but many tasks involve reasoning about relationships between two or more spans of text.
Approach:	They propose a pre-training method that masks contiguous random spans instead of random tokens to train the span boundary representations to predict the entire content of the masked span.
Outcome:	The proposed method outperforms BERT and its better-tuned baselines on span selection tasks and on coreference resolution tasks.

Copied to clipboard

Takuya Uematsu, Hao Wang, Daisuke Kawahara, Tomohide Shibata

A Graph Representation of Semi-structured Data for Web Question Answering (2020.coling-main)

Challenge:	Existing studies to solve QA tasks in an integrated manner are not available in other languages because of the lack of QA datasets.
Approach:	They build a Japanese version of Natural Questions using natural questions from query logs of a search engine and crowdsource it using crowdsourcing.
Outcome:	The proposed datasets are based on natural questions from Japanese search engines and crowdsourced.

Copied to clipboard

Xingyao Zhang, Linjun Shou, Jian Pei, Ming Gong, Lijie Wen, Daxin Jiang

An Interactive Web-Interface for Visualizing the Inner Workings of the Question Answering LSTM (D18-2)

Challenge:	Existing studies treat semi-structured data as flat documents with pieces of text . semi-structural data is more effective to represent rich relational information . question answering is an important feature in most search engines .
Approach:	They propose a graph representation of Web tables and lists based on categorization of components and their relations . they also develop reasoning techniques on the graph model for the question answering task .
Outcome:	The proposed graph improves F1 score by 3.90 points over the state-of-the-art baselines on real datasets.

Copied to clipboard

Ekaterina Loginova, Günter Neumann

AnswerQuest: A System for Generating Question-Answer Items from Multi-Paragraph Documents (2021.eacl-demos)

Challenge:	Existing visualisation methods for deep learning models are limited by their low interpretability and lack a tool for interpreting them.
Approach:	They propose a visualisation tool which plots heatmaps of neurons’ firings and allows a user to check the dependency between neurons and manual features.
Outcome:	The proposed visualisation tool plots heatmaps of neurons’ firings and allows a user to check the dependency between neurons and manual features.

Copied to clipboard

Melissa Roemmele, Deep Sidhpura, Steve DeNeefe, Ling Tsou

Thesis Proposal: LLMs post-training for multilingual medical tasks. Instruction-Tuning, Continual-Pretraining or Reasoning? (2026.acl-srw)

Challenge:	Existing systems that generate and answer questions in a question-and-answer format can facilitate reading comprehension.
Approach:	They propose a system that integrates question answering and question generation tasks to produce a list of Q&A items for a text.
Outcome:	The proposed system generates a catalog of Q&A items for a text.

Copied to clipboard

Pietro Ferrazzi, Alberto Lavelli, Bernardo Magnini

BiQuAD: Towards QA based on deeper text understanding (2021.starsem-1)

Challenge:	Adapting Large Language Models to the medical domain remains an active area of research .
Approach:	They propose to compare three common adaptation approaches to adapt large language models to the medical domain.
Outcome:	The proposed models are built on top of foundational LLMs and rely on different post-training methodologies for domain and task performance.

Copied to clipboard

Frank Grimm, Philipp Cimiano

OpenPI2.0: An Improved Dataset for Entity Tracking in Texts (2024.eacl-long)

Challenge:	Recent question answering and machine reading benchmarks require systems to pinpoint the span of the answer to a given text.
Approach:	They propose a dataset that requires deeper comprehension to answer questions extractively and deductively.
Outcome:	The proposed dataset outperforms existing benchmarks on extractive and deductive questions.

Copied to clipboard

Li Zhang, Hainiu Xu, Abhinav Kommula, Chris Callison-Burch, Niket Tandon

Challenge:	Recent work shows that explicit modeling entity states benefits LMs in procedural tasks.
Approach:	They propose a dataset where entities and attributes are fully canonicalized and additional entity salience annotations are added.
Outcome:	The proposed dataset outperforms existing models on question answering and classical planning tasks.

Shortcomings of Question Answering Based Factuality Frameworks for Error Localization (2023.eacl-main)

Copied to clipboard

Ryo Kamoi, Tanya Goyal, Greg Durrett

RAGthoven: A Configurable Toolkit for RAG-enabled LLM Experimentation (2025.coling-demos)

Challenge:	Abstractive summarization systems often generate summaries with factual errors . many approaches to detect these errors have been proposed, but this capability has not been evaluated in past research .
Approach:	They propose to use question answering-based factuality metrics to detect errors in summaries . they find that QA-based frameworks fail to correctly identify error spans in generated summary .
Outcome:	The proposed methods outperform trivial exact match baselines in localizing errors in summaries.

Copied to clipboard

Gregor Karetka, Demetris Skottis, Lucia Dutková, Peter Hraška, Marek Suppa

Challenge:	Large Language Models (LLMs) have significantly altered the landscape of Natural Language Processing (NLP), but their use as a baseline method has not been extensive.
Approach:	They propose a tool for automatic evaluation of RAG-based pipelines that provides a simple yet powerful abstraction.
Outcome:	The proposed tool provides an automatic evaluation of RAG-based pipelines.

Learning Sparsity for Effective and Efficient Music Performance Question Answering (2025.acl-short)

Copied to clipboard

Xingjian Diao, Tianzhen Yang, Chunhui Zhang, Weiyi Wu, Ming Cheng, Jiang Gui

Learning to Search in Long Documents Using Document Structure (C18-1)

Challenge:	Existing Music AVQA methods rely on dense and unoptimized representations, leading to inefficiencies in the isolation of key information, reduction of redundancy, and prioritization of critical samples.
Approach:	They propose a sparse learning framework specifically designed for Music AVQA to address these challenges.
Outcome:	The proposed framework reduces training time by 28.32% while maintaining accuracy while maintaining state-of-the-art performance on the Music AVQA datasets.

Copied to clipboard

Mor Geva, Jonathan Berant

Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies (2023.eacl-main)

Challenge:	Reading comprehension models are dominated by recurrent neural networks (RNNs) as documents become longer and questions become complex, sequential reading becomes a significant bottleneck.
Approach:	They propose a reading comprehension framework that uses document trees to model an agent that interleaves quick navigation with more expensive answer extraction.
Outcome:	The proposed model improves question answering performance compared to existing models and has a strong information-retrieval baseline.

Copied to clipboard

Md Rizwan Parvez, Jianfeng Chi, Wasi Uddin Ahmad, Yuan Tian, Kai-Wei Chang

KnowShiftQA: How Robust are RAG Systems when Textbook Knowledge Shifts in K-12 Education? (2025.acl-short)

Challenge:	Existing labeled datasets are heavily imbalanced, limiting the QA performance in this domain.
Approach:	They propose a question answering task that captures relevant text segments from unlabeled policy documents and expands the positive examples in the training set.
Outcome:	The proposed framework elevates the baseline by a large margin (10% F1) and achieves a new state-of-the-art F1 score of 50%.

Copied to clipboard

Tianshi Zheng, Weihan Li, Jiaxin Bai, Weiqi Wang, Yangqiu Song

Evaluating Question Answering Evaluation (D19-58)

Challenge:	Existing knowledge discrepancies between textbooks and large language models can undermine RAG systems' performance.
Approach:	They propose to use a dataset to test RAG system robustness against knowledge discrepancies.
Outcome:	The proposed dataset shows that RAG systems suffer performance degradation when faced with knowledge discrepancies.

Copied to clipboard

Anthony Chen, Gabriel Stanovsky, Sameer Singh, Matt Gardner

An Emotional Comfort Framework for Improving User Satisfaction in E-Commerce Customer Service Chatbots (2021.naacl-industry)

Challenge:	Existing n-gram based QA metrics have a number of drawbacks and are not suitable for all extractive tasks.
Approach:	They propose to use BERTScore to evaluate translation for question answering (QA) they also explore whether existing n-gram based metrics are suitable for generative QA .
Outcome:	The proposed BERTScore metric fails to provide stronger correlation with human judgements .

Copied to clipboard

Shuangyong Song, Chao Wang, Haiqing Chen, Huan Chen

IFlyLegal: A Chinese Legal System for Consultation, Law Searching, and Document Analysis (D19-3)

Challenge:	E-commerce has grown rapidly over the last several years, and chatbots for intelligent customer service are simultaneously drawing attention.
Approach:	They propose a framework to obtain proper answer to customers’ emotional questions using emotion classification model and text matching.
Outcome:	The proposed framework is very promising on real online systems.

Copied to clipboard

Ziyue Wang, Baoxin Wang, Xingyi Duan, Dayong Wu, Shijin Wang, Guoping Hu, Ting Liu

WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning (2023.acl-long)

Challenge:	Legal Tech is a system that performs legal consulting, multi-way law searching, and legal document analysis using deep contextual representations and various attention mechanisms.
Approach:	They propose a Chinese legal system that performs legal consulting, multi-way law searching, and legal document analysis using deep contextual representations and various attention mechanisms.
Outcome:	The proposed system performs legal consulting, multi-way law searching, and legal document analysis by exploiting techniques such as deep contextual representations and various attention mechanisms.

Copied to clipboard

Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Sujian Li, Yajuan Lyu

Bend but Don’t Break? Multi-Challenge Stress Test for QA Models (D19-58)

Challenge:	Existing factual consistency metrics are often uncontrollably generating text that is factually inconsistent with inputs.
Approach:	They propose a weakly supervised framework that is directly trained on actual generated samples from language models with weakly annotated labels.
Outcome:	The proposed framework improves on the TRUE benchmark by 3.3% over existing methods with 435M parameters.

Copied to clipboard

Hemant Pugaliya, James Route, Kaixin Ma, Yixuan Geng, Eric Nyberg

Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation (2022.emnlp-main)

Challenge:	a gap remains in reasoning ability compared to a human, and performance tends to degrade when models are exposed to less-constrained tasks.
Approach:	They conduct extensive qualitative and quantitative analyses on the results of four models across four datasets . they relate common errors to model capabilities and discuss a way forward .
Outcome:	The proposed model performance is based on the results of four models across four datasets.

Copied to clipboard

Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Börschinger, Tal Schuster

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies (2021.tacl-1)

Challenge:	despite the importance of question answering, evaluations of QA systems are typically limited by manual annotations . despite this, little progress has been made in QA evaluations based on a single answer .
Approach:	They propose to extend over exact match (EM) with predefined rules or token-level F1 measure . they propose to use a BERT matching measure to approximate QA predictions .
Outcome:	The proposed model improves AE approximations and more accurately reflects the performance of systems.

Copied to clipboard

Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, Jonathan Berant

DeepPavlov: Open-Source Library for Dialogue Systems (P18-4)

Challenge:	Existing questions that explicitly describe the process for deriving the answer are often implicit.
Approach:	They propose a question answering benchmark where the required reasoning steps are implicit in the question and should be inferred using a strategy.
Outcome:	The proposed model is short, topic-diverse, and covers a wide range of strategies.

Copied to clipboard

Mikhail Burtsev, Alexander Seliverstov, Rafael Airapetyan, Mikhail Arkhipov, Dilyara Baymurzina, Nickolay Bushkov, Olga Gureenkova, Taras Khakhulin, Yuri Kuratov, Denis Kuznetsov, Alexey Litinsky, Varvara Logacheva, Alexey Lymar, Valentin Malykh, Maxim Petrov, Vadim Polulyakh, Leonid Pugachev, Alexey Sorokin, Maria Vikhreva, Marat Zaynutdinov

Paraphrasing with Large Language Models (D19-56)

Challenge:	open-source library DeepPavlov is designed for rapid development of dialogue systems.
Approach:	open-source library DeepPavlov is tailored for development of conversational agents . the library prioritizes efficiency, modularity and extensibility with the goal to make it easier to develop dialogue systems from scratch .
Outcome:	the open-source library DeepPavlov is designed for rapid development of dialogue systems . it supports modular as well as end-to-end approaches to implementation of conversational agents .

Copied to clipboard

Sam Witteveen, Martin Andrews

Extractive NarrativeQA with Heuristic Pre-Training (D19-58)

Challenge:	Recent work has shown large language models are adept at text generation and fine-tuning for downstream NLP tasks.
Approach:	They propose a system that generates paraphrased examples in autoregressive fashion using a neural network without the need for techniques such as top-k word selection or beam search.
Outcome:	The proposed system generates paraphrased examples in autoregressive fashion without the need for techniques such as top-k word selection or beam search.

Copied to clipboard

Lea Frermann

Samvaadhana: A Telugu Dialogue System in Hospital Domain (D19-61)

Challenge:	Automated question answering (QA) from text remains a challenge for humans . a striking gap exists between machine and human performance on NLP tasks .
Approach:	They propose a heuristic extractive version of a data set to solve the problem of answer extraction rather than generation.
Outcome:	The proposed model outperforms previous models on summary-level QA from full narratives and on the METEOR metric.

Copied to clipboard

Suma Reddy Duggenpudi, Kusampudi Siva Subrahamanyam Varma, Radhika Mamidi

Goodwill Hunting: Analyzing and Repurposing Off-the-Shelf Named Entity Linking Systems (2021.naacl-industry)

Challenge:	a dialogue system for Hospital domain in Telugu is a resource-poor Dravidian language . the system handles various hospital and doctor related queries .
Approach:	They propose to model a dialogue system for Hospital domain in Telugu which is a resource-poor Dravidian language.
Outcome:	The proposed system achieves a high overall rating and a significantly accurate context-capturing method.

Copied to clipboard

Karan Goel, Laurel Orr, Nazneen Fatema Rajani, Jesse Vig, Christopher Ré

Knowledge Representation

Challenge:	Named entity linking (NEL) is a preprocessing step in commercial systems . a small organization or individual could use an off-the-shelf system to accomplish the same objectives .
Approach:	They examine how to repurpose off-the-shelf NEL systems to correct sport-related errors.
Outcome:	The proposed model can improve sports question-answering accuracy by 25% . the proposed model is based on the best available model .

Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks (2025.acl-srw)

Copied to clipboard

Amir Saeidi, Shivanshu Verma, Md Nayem Uddin, Chitta Baral

OpenT2T: An Open-Source Toolkit for Table-to-Text Generation (2024.emnlp-demo)

Challenge:	Large Language Models (LLMs) excel in math reasoning problemsolving, text generation, summarization, creative writing, among other tasks.
Approach:	They evaluate Direct Preference Optimization and its variants for aligning Large Language Models with human preferences.
Outcome:	The proposed alignment methods achieve near-optimal performance even with smaller subsets of training data.

Copied to clipboard

Haowei Zhang, Shengyun Si, Yilun Zhao, Lujing Xie, Zhijian Xu, Lyuhao Chen, Linyong Nan, Pengcheng Wang, Xiangru Tang, Arman Cohan

Decontextualization: Making Sentences Stand-Alone (2021.tacl-1)

Challenge:	Existing methods for table-to-text generation are limited and benchmarked on a limited number of datasets.
Approach:	They propose to use open-source tools to reproduce existing large language models for performance comparison and expedite the development of new models.
Outcome:	The proposed toolkit compares existing large language models on 9 table-to-text generation datasets and maintains a leaderboard to provide insights for future work.

Copied to clipboard

Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, Michael Collins

Explainability & Interpretability in NLP

Challenge:	Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window.
Approach:	They define a problem of sentence decontextualization by rewriting a sentence to be interpretable out of context while preserving its meaning.
Outcome:	The proposed method can be used in question answering and document understanding tasks.

Towards Transparent Interactive Semantic Parsing via Step-by-Step Correction (2022.findings-acl)

Copied to clipboard

Lingbo Mo, Ashley Lewis, Huan Sun, Michael White

PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs (2024.starsem-1)

Challenge:	Existing studies on semantic parsing focus on mapping a natural-language utterance to a logical form (LF) but natural language may contain ambiguity and variability, making this challenge difficult.
Approach:	They propose an interactive semantic parsing framework that explains the predicted LF step by step in natural language and enables the user to make corrections through natural-language feedback for individual steps.
Outcome:	The proposed framework improves parsing accuracy and transparency in a crowdsourced dialogue dataset.

Copied to clipboard

Ying Su, Jipeng Zhang, Yangqiu Song, Tong Zhang

Simple Question Answering with Subgraph Ranking and Joint-Scoring (N19-1)

Challenge:	Existing approaches to utilizing explicit knowledge graphs (KGs) are limited by the number of nodes in the subgraph.
Approach:	They propose a grounding-pruning-reasoning pipeline to prune noisy nodes in subgraphs to improve the efficiency of graph reasoning with KG.
Outcome:	The proposed method reduces computation cost and memory usage while obtaining decent representation of pruned subgraphs.

Copied to clipboard

Wenbo Zhao, Tagyoung Chung, Anuj Goyal, Angeliki Metallinou

OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)

Challenge:	Knowledge graph based simple question answering is a major area of research in question answering.
Approach:	They propose a framework to describe and analyze existing knowledge graph based simple question answering approaches.
Outcome:	The proposed model achieves a state-of-the-art (85.44% accuracy) on the SimpleQuestions dataset.

Copied to clipboard

Xu Han, Tianyu Gao, Yuan Yao, Deming Ye, Zhiyuan Liu, Maosong Sun

Dissecting GraphRAG: A Modular Analysis of Knowledge Structuring for Factoid Question Answering (2026.tacl-1)

Challenge:	OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch .
Approach:	OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment.
Outcome:	OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment .

Copied to clipboard

Noriki Nishida, Rumana Ferdous Munne, Shanshan Liu, Narumi Tokunaga, Yuki Yamagata, Fei Cheng, Kouji Kozaki, Yuji Matsumoto

Question Decomposition for Retrieval-Augmented Generation (2025.acl-srw)

Challenge:	GraphRAG integrates structured knowledge graphs into question answering . high-quality triple extraction is critical, but lacks granularity and topical coherence . large language models suffer from inherent limitations in their internalized knowledge .
Approach:	They evaluate module-level design choices in GraphRAG for retrieval-augmented generation . they find that triple extraction is critical for accurate and comprehensive retrieval .
Outcome:	The proposed framework outperforms other retrieval-augmented generation frameworks in accuracy and efficiency.

Copied to clipboard

Paul J. L. Ammann, Jonas Golde, Alan Akbik

OpenRT: An Open-source Framework for Reasoning Over Tabular Data (2023.acl-demo)

Challenge:	Retrieval-augmented generation (RAG) is effective for question answering tasks . multi-hop questions, such as "Which company among NVIDIA, Apple, and Google made the biggest profit in 2023?" challenge RAG because relevant facts are often distributed across multiple documents .
Approach:	They propose a pipeline that incorporates question decomposition to ground large language models in verifiable external sources.
Outcome:	The proposed approach improves retrieval and answer accuracy over standard RAG . multi-hop questions often require multiple documents to support the model .

Copied to clipboard

Yilun Zhao, Boyu Mi, Zhenting Qi, Linyong Nan, Minghao Guo, Arman Cohan, Dragomir Radev

AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering (2026.acl-long)

Challenge:	Existing table pre-training methods are benchmarked on a limited number of datasets with varying configurations, resulting in a lack of unified, standardized, fair, and comprehensive comparison between methods.
Approach:	They propose to use OpenRT to reproduce existing table pre-training models and develop new models quickly.
Outcome:	The proposed framework reproduces existing table pre-training models and compares them against four question answering, one fact checking, and one faithful text generation datasets.

Copied to clipboard

Zheyuan Zhang, Kaiwen Shi, Zhengqing Yuan, Zehong Wang, Tianyi Ma, Keerthiram Murugesan, Vincent Galassi, Chuxu Zhang, Yanfang Ye

DESCGEN: A Distantly Supervised Datasetfor Generating Entity Descriptions (2021.acl-long)

Challenge:	Existing approaches to agent routing emphasize cost efficiency while overlooking the fine-grained contextual and relational structure inherent in QA tasks.
Approach:	They propose a framework that formulates multi-agent QA as a knowledge-graph-guided routing problem supervised by empirical performance signals.
Outcome:	The proposed framework outperforms single-agent and ensemble baselines while generalizing across benchmarks and LLM backbones.

Copied to clipboard

Weijia Shi, Mandar Joshi, Luke Zettlemoyer

Knowledge Representation

Challenge:	Short textual descriptions of entities provide summaries of their key attributes but generating entity descriptions can be challenging since information is scattered across multiple sources with varied content and style.
Approach:	They propose to generate an entity summary description from 37K entities from Wikipedia and Fandom, paired with nine evidence documents on average.
Outcome:	The proposed task is entity-centric, more abstractive, and covers a wide range of domains.

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner (2022.findings-naacl)

Copied to clipboard

Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, Dan Roth

LexSym: Compositionality as Lexical Symmetry (2023.acl-long)

Challenge:	Large language models have achieved high performance on various natural language benchmarks, but the explainability of their output remains elusive.
Approach:	They propose an architecture called iterative retrieval-generation reasoner that generates an entailment tree that explains a given hypothesis by using premises from C.
Outcome:	The proposed model outperforms existing benchmarks on premise retrieval and entailment tree generation with around 300% gain in overall correctness.

Copied to clipboard

Ekin Akyurek, Jacob Andreas

One-shot Learning for Question-Answering in Gaokao History Challenge (C18-1)

Challenge:	Existing approaches to generalize compositional models fail to generalise from small datasets.
Approach:	They propose a domain-general and model-agnostic formulation of compositionality as a constraint on symmetries of data distributions rather than models.
Outcome:	The proposed procedure matches or surpasses state-of-the-art, task-specific models on COGS semantic parsing, SCAN and Alchemy instruction following, and CLEVR-CoGenT visual question answering datasets.

Copied to clipboard

Zhuosheng Zhang, Hai Zhao

Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection (N18-1)

Challenge:	Existing work on deep question-answering tasks from admission exams is challenging since it requires effective representation to capture complicated semantic relations between questions and answers.
Approach:	They propose a hybrid neural model for deep question-answering task from history examinations using a gated network and a machine labeler.
Outcome:	The proposed model obtains substantial performance gains over baseline models in terms of multiple evaluation metrics.

Copied to clipboard

Haw-Shiuan Chang, Ziyun Wang, Luke Vilnis, Andrew McCallum

Sailor: Open Language Models for South-East Asia (2024.emnlp-demo)

Challenge:	Existing unsupervised methods for learning hypernyms from unlabeled text are not scaled to large vocabularies or yield unacceptably poor accuracy.
Approach:	They propose an unsupervised method of hypernym discovery using word contexts . they use word2vec to embed word context distributions without supervision .
Outcome:	The proposed method provides double the precision and highest average performance on 11 datasets.

Copied to clipboard

Longxu Dou, Qian Liu, Guangtao Zeng, Jia Guo, Jiahui Zhou, Xin Mao, Ziqi Jin, Wei Lu, Min Lin

Interactive Text Ranking with Bayesian Optimization: A Case Study on Community QA and Summarization (2020.tacl-1)

Challenge:	Large language models (LLMs) rely on English data for training, but are often not comparable across other languages.
Approach:	They propose to develop a family of open language models for SEA languages . they use BPE dropout, aggressive data cleaning and deduplication to improve model robustness .
Outcome:	The proposed models perform well across four benchmarks, including commonsense reasoning, question answering, reading comprehension and examination.

Copied to clipboard

Edwin Simpson, Yang Gao, Iryna Gurevych

Summarization

Challenge:	Existing methods that focus on learning a ranking across the whole candidate space are lacking user or task-specific training data.
Approach:	They propose an interactive ranking approach that actively selects pairs of candidates, from which the user selects the best.
Outcome:	The proposed approach outperforms existing methods in community question answering and extractive multidocument summarization and is an effective reward function for reinforcement learning.

Event Extraction by Answering (Almost) Natural Questions (2020.emnlp-main)

Copied to clipboard

Xinya Du, Claire Cardie

Answer-level Calibration for Free-form Multiple Choice Question Answering (2022.acl-long)

Challenge:	Existing work in event argument extraction relies heavily on entity recognition as a preprocessing/concurrent step, causing error propagation.
Approach:	They propose a question answering task that extracts event arguments in an end-to-end manner.
Outcome:	The proposed framework outperforms prior work on the ACE 2005 task on event argument extraction.

Copied to clipboard

Sawan Kumar

Unseen Entity Handling in Complex Question Answering over Knowledge Base via Language Generation (2021.findings-emnlp)

Challenge:	Pre-trained language models exhibit few-shot and zero-shot learning capability on tasks such as commonsense reasoning.
Approach:	They propose to model context-independent biases in terms of the probability of a choice without the context and to remove it using an unsupervised estimate of similarity with the full context.
Outcome:	The proposed model improves over baselines on commonsense reasoning tasks.

Copied to clipboard

Xin Huang, Jung-Jae Kim, Bowei Zou

It is AI’s Turn to Ask Humans a Question: Question-Answer Pair Generation for Children’s Story Books (2022.acl-long)

Challenge:	Existing methods for complex question answering are limited in the search space of all possible relation paths.
Approach:	They propose a method that directly generates an executable SPARQL query without simplification.
Outcome:	The proposed method significantly outperforms the previous methods and has higher interpretability and computational efficiency than the previous ones.

Copied to clipboard

Bingsheng Yao, Dakuo Wang, Tongshuang Wu, Zheng Zhang, Toby Li, Mo Yu, Ying Xu

Know Better – A Clickbait Resolving Challenge (2022.lrec-1)

Challenge:	Existing question answering (QA) techniques are created mainly to answer questions asked by humans, but in educational applications, teachers often need to decide what questions to ask .
Approach:	They propose to use a fairytale-themed storybook as input to generate QA pairs that can test a student's comprehension skills.
Outcome:	The proposed system outperforms state-of-the-art QAG baseline systems and builds an interactive story-telling application for the future real-world deployment.

Copied to clipboard

Benjamin Hättasch, Carsten Binnig

Long-Context Long-Form Question Answering for Legal Domain (2026.eacl-industry)

Challenge:	a clickbait headline or teaser is used to "bait" the reader into clicking a link to an article . clickbaiting is annoying but effective, and can be countered with specialized models .
Approach:	They propose to construct approaches that can automatically extract relevant information from clickbait articles . they argue that clickbaiting can probably not be defeated with clickbaitting detection alone .
Outcome:	The proposed methods outperform question answering models on clickbait resolving task . the data will be used to give users tools to counter clickbaiting in the future .

Copied to clipboard

Anagha Kulkarni, Parin Rajesh Jhaveri, Prasha Shrestha, Yu Tong Han, Reza Amini, Behrouz Madahian

Towards the First NLP Benchmark for Ladin - an Extremely Low-Resource Language (2026.findings-eacl)

Challenge:	Legal documents have complex document layouts involving multiple nested sections and lengthy footnotes that make question answering challenging.
Approach:	They propose a question answering system that parses document layouts while isolating sections and footnotes and linking them appropriately.
Outcome:	The proposed system can parse complex document layouts while isolating sections and footnotes and linking them appropriately.

Copied to clipboard

Ulin Nuha, Adam Jatowt

Machine Translation

Challenge:	Large language models (LLMs) are limited in low-resource languages due to lack of labeled training data.
Approach:	They propose to use Ladin as a model for sentiment analysis and question answering by incorporating Italian data into machine translation training.
Outcome:	The proposed method improves on existing Italian–Ladin translation baselines.

Generating Complex Question Decompositions in the Face of Distribution Shifts (2025.naacl-long)

Copied to clipboard

Kelvin Han, Claire Gardent

Self-Training for Jointly Learning to Ask and Answer Questions (N18-1)

Challenge:	Question decomposition has been found to improve large language models’ (LLMs) performance on complex question answering (QA) however, performance on the task remains dominated by supervised approaches, suggesting room for making LLMs better decomposers.
Approach:	They propose to generate synthetic decomposition data with only five annotated examples by extending recent advances in using LLM-as-judge and for reranking in novel ways.
Outcome:	The proposed approach generates synthetic decomposition data with only five examples over two benchmark datasets.

Copied to clipboard

Mrinmaya Sachan, Eric Xing

Permutation Invariant Strategy Using Transformer Encoders for Table Understanding (2022.findings-naacl)

Challenge:	Existing methods for question answering and question generation are hard to obtain in many domains.
Approach:	They propose a method for jointly learning to ask and answer questions . they leverage unlabeled text along with labeled question answer pairs for learning .
Outcome:	The proposed method improves on four benchmark datasets on question answering and question generation tasks.

Copied to clipboard

Sarthak Dash, Sugato Bagchi, Nandana Mihindukulasooriya, Alfio Gliozzo

MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering (2020.emnlp-main)

Challenge:	Existing methods for encoding text in tables require additional training and require additional pretraining.
Approach:	They propose a novel encoding strategy that preserves the critical property of permutation invariance across rows or columns.
Outcome:	The proposed approach outperforms state-of-the-art methods on three table interpretation tasks: column type annotation, relation extraction, and entity linking.

Copied to clipboard

Tejas Gokhale, Pratyay Banerjee, Chitta Baral, Yezhou Yang

From Misleading Queries to Accurate Answers: A Three-Stage Fine-Tuning Method for LLMs (2025.findings-acl)

Challenge:	Availability of large-scale datasets has enabled statistical machine learning in vision and language understanding.
Approach:	They propose a training paradigm that exposes models to perceptually similar mutations of input . they show a 10.57% improvement in the VQA-CP challenge .
Outcome:	The proposed training paradigm improves on the visual question answering challenge with 10.57% accuracy.

Copied to clipboard

Guocong Li, Weize Liu, Yihang Wu, Ping Wang, Shuaihan Huang, Hongxia Xu, Jian Wu

Learning Rich Representation of Keyphrases from Text (2022.findings-naacl)

Challenge:	Existing methods focus on correcting the output but overlook the ability of LLMs to detect and correct misleading content in the input itself.
Approach:	They propose a three-stage fine-tuning method that improves LLMs' ability to detect and correct misleading information in input queries.
Outcome:	The proposed method improves accuracy and factuality of LLM responses while also reducing hallucinations.

Copied to clipboard

Mayank Kulkarni, Debanjan Mahata, Ravneet Arora, Rajarshi Bhowmik

ExAnte: A Benchmark for Ex-Ante Inference in Large Language Models (2026.eacl-long)

Challenge:	Prior work has referred to extractive (part of document) or abstractive (not part of document).
Approach:	They propose to use a new pre-training objective to introduce keyphrases into transformer language models in discriminative and generative settings.
Outcome:	The proposed model improves performance in discriminative and generative settings and also improves on named entity recognition, question answering, relation extraction and abstractive summarization tasks.

Copied to clipboard

Yachuan Liu, Xiaochun Wei, Lin Shi, Xinnuo Li, Bohan Zhang, Paramveer Dhillon, Qiaozhu Mei

Enhancing Financial Table and Text Question Answering with Tabular Graph and Numerical Reasoning (2022.aacl-main)

Challenge:	Large language models (LLMs) struggle with ex-ante reasoning—making inferences or predictions without access to future information.
Approach:	They propose a benchmark that assesses LLMs’ ex-ante inference ability across four tasks: stock prediction, question answering, Wikipedia event generation, and scientific publication generation.
Outcome:	The proposed benchmark assesses LLMs’ ex-ante inference ability across four tasks.

Copied to clipboard

Rungsiman Nararatwong, Natthawut Kertkeidkachorn, Ryutaro Ichise

AQuAECHR: Attributed Question Answering for European Court of Human Rights (2025.findings-acl)

Challenge:	Existing models that learn tabular structures in financial documents do not understand tables and numbers.
Approach:	They propose to infuse explicit tabular structures through a graph neural network to improve model's performance in question answering.
Outcome:	The proposed model outperforms the baseline model in low-resource settings while outperforming the graph module.

Copied to clipboard

Korbinian Q. Weidinger, Santosh T.y.s.s, Oana Ichim, Matthias Grabmair

Recipe Instruction Semantics Corpus (RISeC): Resolving Semantic Structure and Zero Anaphora in Recipes (2020.aacl-main)

Challenge:	LLMs are widely used for information seeking, but their generated responses often suffer from hallucinations, hindering their widespread adoption in high stakes domains such as law.
Approach:	They propose to attribute legal question answering to an actual source to improve factuality and verifiability of the answer.
Outcome:	The proposed framework improves the factuality and verifiability of legal question answering by combining a dataset from ECHR case law guides with an LLM-based filtering pipeline.

Copied to clipboard

Yiwei Jiang, Klim Zaporojets, Johannes Deleu, Thomas Demeester, Chris Develder

Seeing the wood for the trees: a contrastive regularization method for the low-resource Knowledge Base Question Answering (2022.findings-naacl)

Challenge:	Existing approaches to understanding recipe instructions make assumptions that are domain specific.
Approach:	They propose a new dataset for information extraction on recipes . they avoid a priori pre-defining domain-specific predicates to recognize . instead, they focus on basic understanding of the expressed semantics .
Outcome:	The proposed dataset avoids a priori pre-defining domain-specific predicates to recognize . instead, it focuses on basic understanding of the expressed semantics rather than reducing them to a simplified state representation.

Copied to clipboard

Jpliu@wtu.edu.cn Jpliu@wtu.edu.cn, Shijie Mei, Xinrong Hu, Xun Yao, Jack Yang, Yi Guo

A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation (2020.acl-main)

Challenge:	Existing methods for Knowledge Base Question Answering rely on semantic parsing and information retrieval.
Approach:	They propose a contrastive regularization based method to extract correct answer entities from a context knowledge base and a corresponding question.
Outcome:	The proposed method achieves state-of-the-art performance on the WebQuestionsSP dataset and the effectiveness of proposed modules is also evaluated.

Copied to clipboard

Jan Deriu, Katsiaryna Mlynchyk, Philippe Schläpfer, Alvaro Rodrigo, Dirk von Grünigen, Nicolas Kaiser, Kurt Stockinger, Eneko Agirre, Mark Cieliebak

End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions (P18-2)

Challenge:	Existing methods to efficiently construct corpus for question answering over structured data are time-consuming and cost-intensive.
Approach:	They propose a method to efficiently construct a corpus for question answering over structured data.
Outcome:	The proposed method triples the annotation speed while maintaining complexity of queries.

Copied to clipboard

Wenpeng Yin, Dan Roth, Hinrich Schütze

Awakening Augmented Generation: Learning to Awaken Internal Knowledge of Large Language Models for Question Answering (2025.coling-main)

Challenge:	Existing datasets for textual entailment (TE) have been used to study TE.
Approach:	They propose a deep explorations of inter-sentence interactions for textual entailment task that uses a convolution to make important words in P and H play a dominant role in learnt representations.
Outcome:	Experiments show that the pretrained DEISTE on SciTail gets 5% improvement over prior state of the art and that it generalizes well on RTE-5.

Copied to clipboard

Huanxuan Liao, Shizhu He, Yao Xu, Yuanzhe Zhang, Shengping Liu, Kang Liu, Jun Zhao

Elaboration-Generating Commonsense Question Answering at Scale (2023.acl-long)

Challenge:	Recent studies indicate that Large Language Models model rich knowledge, but it is often not activated and awakened.
Approach:	They propose a framework that leverages richer context to enhance question answering . Explicit awakening fine-tunes a context generator to create a synthetic, compressed document that functions as symbolic context.
Outcome:	The proposed framework mimics the human ability to answer questions using only thinking and recalling to compensate for knowledge gaps.

Copied to clipboard

Wenya Wang, Vivek Srikumar, Hannaneh Hajishirzi, Noah A. Smith

HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data (2020.findings-emnlp)

Challenge:	elaborations are generated using language models that generate background knowledge that helps improve performance . human evaluations show that the quality of the generated ellaborations is high .
Approach:	They propose to finetune smaller language models to generate useful intermediate context . they compare a language model with an answer predictor and generate elaborations . human evaluations show that the quality of the generated ellaborations is high .
Outcome:	The proposed framework outperforms other models on commonsense questions on four commons sense benchmarks.

Copied to clipboard

Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, William Yang Wang

Optimizing Retrieval-augmented Reader Models via Token Elimination (2023.emnlp-main)

Challenge:	Existing question answering datasets focus on dealing with homogeneous information, but using homogenous information alone might lead to coverage problems.
Approach:	They propose a large-scale question-answering dataset that requires reasoning on heterogeneous information.
Outcome:	The proposed model can achieve an EM score of 40% while the existing model is far behind human performance.

Copied to clipboard

Moshe Berchansky, Peter Izsak, Avi Caciularu, Ido Dagan, Moshe Wasserblat

Efficient Out-of-Domain Detection for Sequence to Sequence Models (2023.findings-acl)

Challenge:	Existing methods for ODQA use a retrieval-augmented language model . a generative model can cause a significant bottleneck in decoding time .
Approach:	They propose to eliminate some of the retrieved information that might not contribute essential information to the answer generation process.
Outcome:	The proposed method reduces run-time by up to 62.2% with only 2% reduction in performance and improves performance.

Copied to clipboard

Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, Artem Shelmanov

LLaSA: Large Language and Structured Data Assistant (2025.naacl-long)

Challenge:	Sequence-to-sequence (seq2sequ) models are a ubiquitous tool for text generation but they are not suitable for many other tasks.
Approach:	They propose to use UE techniques to identify out-of-domain (OOD) inputs where the model is susceptible to errors.
Outcome:	The proposed methods outperform heavyweight ensembles on the task of OOD detection.

Copied to clipboard

Yao Xu, Shizhu He, Jiabei Chen, ZengXiangrong ZengXiangrong, Bingning Wang, Guang Liu, Jun Zhao, Kang Liu

Robust Question Answering Through Sub-part Alignment (2021.naacl-main)

Challenge:	Structured knowledge grounding (SKG) tasks are a key part of many NLP applications.
Approach:	They propose a framework for enhancing LLMs' ability to handle structured data . they represent various types of structured data in a unified hypergraph format .
Outcome:	The proposed framework outperforms existing methods on SKG tasks using LoRA finetuning.

Copied to clipboard

Jifan Chen, Greg Durrett

On the Transferability of Minimal Prediction Preserving Inputs in Question Answering (2021.naacl-main)

Challenge:	Current textual question answering models fail to generalize to out-of-domain settings.
Approach:	They propose to decompose question and context into smaller units and align them to find the answer.
Outcome:	The proposed model is more robust than the standard BERT QA model on adversarial and out-of-domain datasets.

Copied to clipboard

Shayne Longpre, Yi Lu, Chris DuBois

A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the CHATGPT Era and Beyond (2024.eacl-long)

Challenge:	Recent work establishes the presence of short, uninterpretable input fragments that yield high confidence and accuracy in neural models.
Approach:	They investigate competing hypotheses for the existence of MPPIs in question answering . they discover a perplexing invariance of MPIs to random training seed, model architecture, pretraining, and training domain.
Outcome:	The proposed model performance is higher than comparable short queries.

Copied to clipboard

Abhinav Ramesh Kashyap, Thanh-Tung Nguyen, Viktor Schlegel, Stefan Winkler, See-Kiong Ng, Soujanya Poria

MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation (2026.acl-long)

Challenge:	Sentence representations are a critical component in NLP applications such as retrieval, question answering, and text classification.
Approach:	They present a systematic review of the literature on sentence representations focusing mostly on deep learning models.
Outcome:	The proposed methods highlight the key contributions and challenges in this area and suggest potential avenues for improving the quality and efficiency of sentence representations.

Copied to clipboard

Mehul Agarwal, Aditya Aggarwal, Arnav Goel, Medha Hira, Anubha Gupta

Syntactic Text Processing

Challenge:	In morphologically rich languages, gender influences verb conjugation, pronouns, and even first-person constructions with explicit and implicit mentions of gender.
Approach:	They propose a morphologically grounded large-scale benchmark dataset for evaluating gender-aware generation in three typologically diverse grammatically gendered languages: French, Arabic, and Hindi.
Outcome:	The proposed dataset compares 15 popular multilingual large language models on their ability to handle morphological gender and morphology agreement.

Narrative Embedding: Re-Contextualization Through Attention (2021.emnlp-main)

Copied to clipboard

Sean Wilner, Daniel Woolridge, Madeleine Glick

Drop Dropout on Single Epoch Language Model Pretraining (2025.findings-acl)

Challenge:	a novel approach to narrative event representation uses attention to re-contextualize events across the whole story . a recent study shows that attention is used to attach event semantics to tokens .
Approach:	They propose an unsupervised approach to narrative event representation using attention to re-contextualize events across the whole story.
Outcome:	The proposed approach achieves state of the art performance on multiple choice and story cloze tasks.

Copied to clipboard

Houjun Liu, John Bauer, Christopher D Manning

D2S: Document-to-Slide Generation Via Query-Based Text Summarization (2021.naacl-main)

Challenge:	Initial dropout was seen as a breakthrough regularization technique that reduced overfitting, yet single-epoch pretraining tasks common to modern LLMs yield minimal overfit.
Approach:	They propose to use dropout during single-epoch pretraining to reduce overfitting in language modeling, morpho-syntax, question answering, and MNLI to improve performance.
Outcome:	The results show that dropout is not used in large LLMs and improves performance in language modeling, morpho-syntax, question answering, and MNLI.

Copied to clipboard

Edward Sun, Yufang Hou, Dakuo Wang, Yunfeng Zhang, Nancy X. R. Wang

Bidirectional Reasoning Supervision for Multilingual Financial Decision Making (2025.emnlp-industry)

Challenge:	Existing research efforts to automate the document-to-slide generation process face a critical challenge: no publicly available dataset for training and benchmarking.
Approach:	They propose a dataset SciDuet that gathers papers and their corresponding slides from recent years’ NLP and ML conferences.
Outcome:	The proposed system outperforms state-of-the-art summarization baselines on both automated ROUGE metrics and qualitative human evaluation.

Copied to clipboard

Muhammad Rafsan Kabir, Jawad Ibn Ahad, Robin Krambroeckers, Silvia Ahmed, M M Lutfe Elahi, Nabeel Mohammed, Shafin Rahman

LLMs Cannot (Yet) Match the Specificity and Simplicity of Online Communities in Long Form Question Answering (2024.findings-emnlp)

Challenge:	Large Language Models have been used for sentiment analysis, machine translation, and question answering, but their effectiveness in the multilingual financial domain remains unknown.
Approach:	They propose a fine-tuning approach that integrates positive and negative rationales alongside classification labels.
Outcome:	The proposed approach outperforms existing methods across English, Hindi, Bengali, and Telugu, and is suitable for industry applications.

Copied to clipboard

Kris-Fillip Kahl, Tolga Buz, Russa Biswas, Gerard De Melo

The Context-Dependent Additive Recurrent Neural Net (N18-1)

Challenge:	Recent years have positioned Large Language Models (LLMs) as powerful question answering (QA) tools, shifting users away from interacting in communities towards discourse with AI-driven conversational interfaces.
Approach:	They propose to use a QA preference dataset to fine-tune and align Large Language Models (LLMs) from more than 7.4 million submissions and 82 million comments from 2008 to 2022 in Reddit’s 15 largest finance communities.
Outcome:	The proposed framework improves on the social quality of the data, and the proposed framework is more accurate and more specific.

Copied to clipboard

Quan Hung Tran, Tuan Lai, Gholamreza Haffari, Ingrid Zukerman, Trung Bui, Hung Bui

AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor (2026.findings-acl)

Challenge:	Contextual sequence mapping is one of the fundamental problems in Natural Language Processing (NLP).
Approach:	They propose a new family of Recurrent Neural Networks that address contextual sequence mapping . they propose to use contextual signals to control the flow of information .
Outcome:	The proposed architecture outperforms existing methods on dialog problem and language model . the proposed architectures are based on a novel family of recurrent neural networks .

Copied to clipboard

Shu Yang, Jingyu Hu, Tong Li, Hanqi Yan, Wenxuan Wang, Di Wang

Challenge:	AutoMonitor-Bench evaluates the reliability of LLM-based misbehavior monitors across diverse tasks and failure modes.
Approach:	They introduce AutoMonitor-Bench, a benchmark designed to evaluate misbehavior monitors across diverse tasks and failure modes.
Outcome:	The new benchmark evaluates the reliability of LLM-based misbehavior monitors across tasks and failure modes.

DrugEHRQA: A Question Answering Dataset on Structured and Unstructured Electronic Health Records For Medicine Related Queries (2022.lrec-1)

Copied to clipboard

Jayetri Bardhan, Anthony Colas, Kirk Roberts, Daisy Zhe Wang

Texts or Images? A Fine-grained Analysis on the Effectiveness of Input Representations and Models for Table Question Answering (2025.findings-acl)

Challenge:	a new question answering dataset is being developed for electronic health records . structured tables and unstructured notes can be duplicated, contradictory or provide additional context .
Approach:	They develop a question-answer-matching dataset using structured tables and unstructured notes from an EHR.
Outcome:	The proposed model is based on a model with a modality selection network . it uses the prediction of a RAT-SQL to choose between EHR tables and clinical notes .

Copied to clipboard

Wei Zhou, Mohsen Mesgar, Heike Adel, Annemarie Friedrich

Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021.acl-short)

Challenge:	Existing studies on table question answering use either serialized text or images . lack of controlled setups limits fine-grained distinctions between these approaches.
Approach:	They propose a method selecting table representations dynamically and compare their performance to previous studies.
Outcome:	The proposed method improves performance 10% compared to using both representations indiscriminately.

Copied to clipboard

Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang

Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models (2024.findings-naacl)

Challenge:	Recent work focuses on question answering based on machine reading comprehension . current approaches treat QA as extracting a consecutive piece of text to a given question.
Approach:	They propose a generative QA model that incorporates an extractive mechanism into a model.
Outcome:	The proposed model improves quality and semantic accuracy over baseline models.

Copied to clipboard

Fobo Shi, Peijun Qing, Dong Yang, Nan Wang, Youbo Lei, Haonan Lu, Xiaodong Lin, Duantengchuan Li

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (2021.emnlp-main)

Challenge:	Prompt engineering is an essential technique for enhancing the abilities of large language models (LLMs) by providing explicit and specific instructions.
Approach:	They propose a new approach that uses text embeddings to obtain basis vectors by matrix decomposition and constructs a space for representing all prompts.
Outcome:	The proposed approach significantly outperforms state-of-the-art prompt paradigms on ten public reasoning benchmarks.

Copied to clipboard

Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang, Saksham Singhal, Xian-Ling Mao, Heyan Huang, Xia Song, Furu Wei

Nonparametric Masked Language Modeling (2023.findings-acl)

Challenge:	Multilingual T5 pretrains a sequence-to-sequence model on monolingual texts, but it has shown promising results on many cross-lingual tasks.
Approach:	They propose a partially non-autoregressive objective for text-to-text pre-training and propose mT6 to improve cross-lingual transferability over multilingual T5.
Outcome:	The proposed model improves cross-lingual transferability over existing models.

Copied to clipboard

Sewon Min, Weijia Shi, Mike Lewis, Xilun Chen, Wen-tau Yih, Hannaneh Hajishirzi, Luke Zettlemoyer

Crafting In-context Examples according to LMs’ Parametric Knowledge (2024.findings-naacl)

Challenge:	Existing language models (LMs) predict tokens with a softmax over a finite vocabulary, which can make it difficult to predict rare tokens or phrases.
Approach:	They introduce a nonparametric masked language model that replaces a softmax with a distribution over every phrase in a reference corpus and uses an in-batch approximation to train it.
Outcome:	The proposed model outperforms larger parametric models on 16 tasks including classification, fact probing and question answering.

Copied to clipboard

Yoonsang Lee, Pranav Atreya, Xi Ye, Eunsol Choi

Semantic Structure Based Query Graph Prediction for Question Answering over Knowledge Graph (2022.coling-1)

Challenge:	In-context examples can improve the performance of knowledge-rich tasks such as question answering by triggering a language model to surface information stored in its parametric knowledge.
Approach:	They propose to construct in-context example sets based on model's parametric knowledge by prompting models with 'unknown' examples.
Outcome:	The proposed model can perform better on in-context examples in three multi-answer question answering datasets, and prompting with ‘unknown’ examples decreases the performance.

Copied to clipboard

Mingchen Li, Shihao Ji

Exploiting Numerical-Contextual Knowledge to Improve Numerical Reasoning in Question Answering (2022.findings-naacl)

Challenge:	Existing approaches for query graph generation ignore the semantic structure of a question . Existing methods ignore the structure of the question, resulting in noisy query graph candidates.
Approach:	They propose to build query graphs from natural language questions to predict semantic structure of a question.
Outcome:	The proposed method can predict the semantic structure of a question using six semantic structures from common questions in KGQA.

Copied to clipboard

Jeonghwan Kim, Junmo Kang, Kyung-min Kim, Giwon Hong, Sung-Hyon Myaeng

Supervised and Unsupervised Transfer Learning for Question Answering (N18-1)

Challenge:	Existing numerical reasoning models overly rely on parametric knowledge at inference time . previous studies show that understanding numbers in text improves numerical reasoning accuracy .
Approach:	They propose a numerical reasoning model that leverages parametric knowledge to alleviate this over-reliance on parametric information.
Outcome:	The proposed model improves numerical reasoning accuracy and performance in DROP.

Copied to clipboard

Yu-An Chung, Hung-Yi Lee, James Glass

XLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction (2022.findings-naacl)

Challenge:	Several QA scenarios and datasets have been introduced over the past few years.
Approach:	They conduct extensive experiments to investigate the transferability of knowledge from a source QA dataset to a target dataset using two QA models.
Outcome:	The proposed model outperforms the previous best model on TOEFL listening comprehension test by 7% on target datasets.

Copied to clipboard

Yuwei Cao, William Groves, Tanay Kumar Saha, Joel Tetreault, Alejandro Jaimes, Hao Peng, Philip Yu

Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications (P19-1)

Challenge:	Temporal Expression Extraction (TEE) is essential for understanding time in natural language.
Approach:	They propose a framework for multilingual Temporal Expression Extraction that leverages pre-trained language models to prompt cross-language knowledge transfer from English to non-English languages.
Outcome:	The proposed framework outperforms the existing SOTA methods on French, Spanish, Portuguese, and Basque by large margins.

Copied to clipboard

Wei Zhao, Haiyun Peng, Steffen Eger, Erik Cambria, Min Yang

Challenge:	Existing approaches to generalize deep neural networks are datahungry and generalize poorly from small datasets.
Approach:	They propose an agreement score to evaluate routing processes at instance-level and an adaptive optimizer to enhance routing.
Outcome:	The proposed approach improves on two NLP tasks and in low-resource settings with few training instances.

Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization (P19-1)

Copied to clipboard

Paul Pu Liang, Zhun Liu, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhutdinov, Louis-Philippe Morency

On Context Utilization in Summarization with Large Language Models (2024.acl-long)

Challenge:	Existing methods to regularize multimodal data are imperfect due to imperfect modalities, missing entries or noise corruption.
Approach:	They propose a method to regularize multimodal data by tensor rank minimization . they use correlations between time and modalities to generate low-rank tenses .
Outcome:	The proposed model achieves good results across various levels of imperfection.

Copied to clipboard

Mathieu Ravaut, Aixin Sun, Nancy Chen, Shafiq Joty

Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts (2025.emnlp-industry)

Challenge:	Large language models excel in abstractive summarization tasks, delivering fluent and pertinent summaries.
Approach:	They conduct the first comprehensive study on context utilization and position bias in summarization.
Outcome:	The proposed benchmark compares two methods to alleviate position bias in summarization tasks.

Copied to clipboard

Anwesan Pal, Karen Hovsepian, Tinghao Guo, Mengnan Zhao, Somendra Tripathi, Nikos Kanakaris, George Mihaila, Sumit Nigam

Syntactic Text Processing

Challenge:	Recent studies into effective context lengths of flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models.
Approach:	They propose a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios without degrading and altering the integrity and composition of retrieved documents.
Outcome:	The proposed strategy boosts performance in long-context scenarios without degrading and altering the integrity and composition of retrieved documents.

A Neural-Symbolic Approach to Natural Language Understanding (2022.findings-emnlp)

Copied to clipboard

Zhixuan Liu, Zihao Wang, Yuan Lin, Hang Li

Towards Few-shot Fact-Checking via Perplexity (2021.naacl-main)

Challenge:	Pre-trained language models have enabled deep neural networks to perform natural language understanding tasks, but their performance can drastically deteriorate when logical reasoning is needed.
Approach:	They propose a framework for NLU based on analogical reasoning based upon neural processing and logical reasoning using both neural and symbolic processing.
Outcome:	The proposed framework outperforms state-of-the-art methods on two NLU tasks, question answering (QA) and natural language inference (NLI).

Copied to clipboard

Nayeon Lee, Yejin Bang, Andrea Madotto, Pascale Fung

Efficient and Robust Question Answering from Minimal Context over Documents (P18-1)

Challenge:	Recent studies have shown that pre-trained language models can perform few-shot learning for various downstream tasks, such as question answering and machine translation.
Approach:	They propose a method to leverage the powerful transfer learning ability of a language model via a perplexity score to learn few-shot for the fact-checking task.
Outcome:	The proposed method outperforms the Major Class baseline by 10% on the F1-Macro metric across multiple datasets.

Copied to clipboard

Sewon Min, Victor Zhong, Richard Socher, Caiming Xiong

BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biology (2023.emnlp-main)

Challenge:	Recent work shows that neural QA models are sensitive to adversarial inputs.
Approach:	They propose a sentence selector to select the minimal set of sentences to feed into a QA model.
Outcome:	The proposed system reduces training time and inference time by up to 13 times . it is comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQu AD-Open .

Copied to clipboard

Odhran O’Donoghue, Aleksandar Shtedritski, John Ginger, Ralph Abboud, Ali Ghareeb, Samuel Rodriques

SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generation (2025.findings-acl)

Challenge:	Large Language Models (LLMs) have impressive capabilities on a wide range of tasks, such as question answering and the generation of coherent text and code.
Approach:	They propose a framework for automatic evaluation of large language models on open-ended planning problems and a dataset of biology protocols with corresponding pseudocode representations.
Outcome:	The proposed framework evaluates an LLM on a dataset of biology protocols with corresponding pseudocode representations.

Copied to clipboard

Yuzheng Cai, Zhenyue Guo, YiWen Pei, WanRui Bian, Weiguo Zheng

TABi: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval (2022.findings-acl)

Challenge:	Recent advances in large language models (LLMs) have shown impressive versatility across various tasks.
Approach:	They propose a retrieval-augmented generation method that integrates LLMs with external knowledge sources to produce grounded outputs.
Outcome:	The proposed method outperforms state-of-the-art KG-driven methods in question answering and fact verification.

Copied to clipboard

Megan Leszczynski, Daniel Fu, Mayee Chen, Christopher Re

LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data (2025.findings-acl)

Challenge:	Existing methods for entity retrieval use mention boundaries but unstructured text . state-of-the-art methods struggle to retrieve rare entities for ambiguous mentions .
Approach:	They propose a method to jointly train bi-encoders on knowledge graph types and unstructured text for entity retrieval for open-domain tasks.
Outcome:	The proposed method improves retrieval of rare entities on Ambiguous Entity Retrieval sets while maintaining strong overall retrieval performance on open-domain tasks.

Copied to clipboard

Cehao Yang, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Shengjie Ma, Aofan Liu, Hui Xiong, Jian Guo

The APVA-TURBO Approach To Question Answering in Knowledge Base (C18-1)

Challenge:	Long-context processing ability has emerged as a significant challenge for large language models.
Approach:	They propose a pipeline for synthesizing faithful long-context reasoning instruction datasets . they integrate ground truth and citation-based reasoning prompts integrating them .
Outcome:	The proposed pipeline eliminates distractions and improves reasoning chains.

Copied to clipboard

Yue Wang, Richong Zhang, Cheng Xu, Yongyi Mao

Who Taught You That? Tracing Teachers in Model Distillation (2025.findings-acl)

Challenge:	Existing query languages for question answering over knowledge bases are not capable of processing queries presented in human language directly.
Approach:	They advocate a new model architecture that includes a verification mechanism for checking the correctness of predicted relations.
Outcome:	The proposed approach dramatically improves the question answering performance.

Copied to clipboard

Somin Wadhwa, Chantal Shaib, Silvio Amir, Byron C Wallace

MAFiD: Moving Average Equipped Fusion-in-Decoder for Question Answering over Tabular and Textual Data (2023.findings-eacl)

Challenge:	Xu et al., 2006, show that model distillation can imbue efficient small language models with task-specific capabilities competitive with expensive teacher LLMs.
Approach:	They propose to distill outputs from a large teacher model to a small student model . they propose to use part-of-speech templates as higher-order linguistic features capable of capturing distinctive signals from teacher models that persist in distilled student outputs.
Outcome:	The proposed model distillation technique can imbue efficient small language models with task-specific capabilities competitive with (expensive) teacher LLMs.

Copied to clipboard

Sung-Min Lee, Eunhwan Park, Daeryong Seo, Donghyeon Jeon, Inho Kang, Seung-Hoon Na

Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia (C18-1)

Challenge:	Experimental results show that Transformer-based questions have a "long" hybrid sequence over tabular and textual elements, causing long-range reasoning problems.
Approach:	They propose a moving average-equipped fusion-in-decoder to handle long-range reasoning problems . they use FiD and EMA to combine different levels of reasoning .
Outcome:	Experimental results show that the proposed model increases exact matching and F1 by 1.1 and 1.7 on the blind test set.

Copied to clipboard

Michael Azmy, Peng Shi, Jimmy Lin, Ihab Ilyas

Sources of Hallucination by Large Language Models on Inference Tasks (2023.findings-emnlp)

Challenge:	Existing datasets for question answering over knowledge graphs lack answer triples from Freebase . a defunct knowledge graph makes it difficult to build "real-world" question answering systems .
Approach:	They propose a benchmark dataset for simple question answering over knowledge graphs that maps SimpleQuestions entities and predicates from Freebase to DBpedia.
Outcome:	The proposed dataset provides simple yet strong baselines with and without neural networks.

Copied to clipboard

Nick McKenna, Tianyi Li, Liang Cheng, Mohammad Hosseini, Mark Johnson, Mark Steedman

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (C18-1)

Challenge:	Large Language Models (LLMs) are claimed to be capable of Natural Language Inference (NLI)
Approach:	They propose to use LLMs to probe their behavior using controlled experiments.
Outcome:	The proposed models perform significantly worse on NLI test samples which do not conform to these biases than those which do.

Copied to clipboard

Vikas Yadav, Steven Bethard

Beyond Memorization: The Challenge of Random Memory Access in Language Models (2024.acl-long)

Challenge:	Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc.
Approach:	They propose to use recurrent neural networks to generate NERs over characters, sub-words and/or word embeddings to improve named entity recognition.
Outcome:	The proposed architectures are better than those based on feature engineering and other supervised or semi-supervised learning algorithms.

Copied to clipboard

Tongyao Zhu, Qian Liu, Liang Pang, Zhengbao Jiang, Min-Yen Kan, Min Lin

QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization (2022.naacl-main)

Challenge:	Recent advances in Language Models (LMs) have shown their effectiveness in knowledge-intensive tasks.
Approach:	They investigate whether a generative language model is able to access its memory sequentially or randomly.
Outcome:	The proposed LMs are able to access memory sequentially or randomly.

Copied to clipboard

Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, Caiming Xiong

Calibrating Structured Output Predictors for Natural Language Processing (2020.acl-main)

Challenge:	Existing studies on text summarization factual consistency are divided into two categories . entailment-based and question answering-based metrics are the most efficient .
Approach:	They propose an optimized QA-based metric that improves factual consistency by 14% . they compare entailment-based and QA metrics to find the best fit .
Outcome:	The proposed metric outperforms the best performing entailment-based metric on the SummaC factual consistency benchmark.

Copied to clipboard

Abhyuday Jagannatha, Hong Yu

Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understanding (2025.findings-naacl)

Challenge:	Several modern machine-learning based NLP systems can provide a confidence score with their output predictions.
Approach:	They propose a general calibration scheme for output entities of interest in NLP applications that can be used to calibrate confidence scores.
Outcome:	The proposed calibration scheme outperforms current calibration techniques for Named Entity Recognition, Part-of-speech tagging and Question Answering systems.

Copied to clipboard

Xingjian Diao, Chunhui Zhang, Weiyi Wu, Zhongyu Ouyang, Peijun Qing, Ming Cheng, Soroush Vosoughi, Jiang Gui

COMMIT: Code-Mixing English-Centric Large Language Model for Multilingual Instruction Tuning (2024.findings-naacl)

Challenge:	Multimodal foundation models have demonstrated significant success in tasks such as visual captioning, question answering, and image-text retrieval.
Approach:	They propose a specialized cognitive module, temporal working memory, which selectively retains task-relevant information across temporal dimensions.
Outcome:	The module retains task-relevant information across temporal dimensions, ensuring that critical details are preserved throughout the processing of video and audio content.

Copied to clipboard

Jaeseong Lee, YeonJoon Jung, Seung-won Hwang

Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs (C18-1)

Challenge:	COMMIT improves the exact match score of low-resource language QA by 32x.
Approach:	They propose to specialize instruction tuning to deviate from English-centric instruction tuning . they propose to perform cross-lingual alignment to overcome data imbalance .
Outcome:	The proposed method improves the exact match score of low-resource language QA by 32x.

Copied to clipboard

Wenpeng Yin, Yadollah Yaghoobzadeh, Hinrich Schütze

MetaICL: Learning to Learn In Context (2022.naacl-main)

Challenge:	Large scale knowledge graphs (KGs) such as Freebase are generally incomplete.
Approach:	They propose a model that predicts entities at each step of mh-KB paths . the model is based on recurrent neural networks and vector representations of entities and relations .
Outcome:	The proposed models show state-of-the-art for two important multi-hop KG reasoning tasks.

Copied to clipboard

Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi

Sequence-to-Sequence Knowledge Graph Completion and Question Answering (2022.acl-long)

Challenge:	Large language models can do in-context learning by conditioning on a few training examples with no parameter updates or task-specific templates.
Approach:	They propose a meta-training framework where a pretrained language model is tuned to do in-context learning on a large set of training tasks.
Outcome:	The proposed framework outperforms baseline models on 142 NLP datasets and a range of target tasks with domain shifts.

Copied to clipboard

Apoorv Saxena, Adrian Kochsiek, Rainer Gemulla

Deep Contextualized Word Representations (N18-1)

Challenge:	Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embeddable vectors.
Approach:	They propose to use an off-the-shelf encoder-decoder Transformer model to generate a knowledge graph embedding model that can be used for KG link prediction and incomplete KG question answering.
Outcome:	The proposed model outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.

Copied to clipboard

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer

Graph Reasoning for Question Answering with Triplet Retrieval (2023.findings-acl)

Challenge:	a new type of deep contextualized word representation is proposed for language understanding problems . word vectors are learned functions of the internal states of a deep bidirectional language model .
Approach:	They propose a new type of deep contextualized word representation that models complex features of word use and how they vary across linguistic contexts.
Outcome:	The proposed representations improve the state of the art across six challenging NLP problems.

Copied to clipboard

Shiyang Li, Yifan Gao, Haoming Jiang, Qingyu Yin, Zheng Li, Xifeng Yan, Chao Zhang, Bing Yin

Interactive Machine Comprehension with Information Seeking Agents (2020.acl-main)

Challenge:	Existing methods to answer complex questions require reasoning over knowledge graphs (KGs) state-of-the-art methods constrain retrieved knowledge in local subgraphs and discard more diverse triplets that are disconnected but useful for question answering.
Approach:	They propose a method to retrieve the most relevant triplets from KGs and then rerank them, which are then concatenated with questions to be fed into language models.
Outcome:	The proposed method outperforms state-of-the-art methods on commonsenseQA and OpenbookQA datasets with 4.6% absolute accuracy.

Copied to clipboard

Xingdi Yuan, Jie Fu, Marc-Alexandre Côté, Yi Tay, Chris Pal, Adam Trischler

Unsupervised Improvement of Factual Knowledge in Language Models (2023.eacl-main)

Challenge:	Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA).
Approach:	They propose a method that reframes existing machine reading comprehension (MRC) datasets as interactive, partially observable environments.
Outcome:	The proposed method "occludes" the majority of a document’s text and adds context-sensitive commands that reveal "glimpses" of the hidden text to a model.

Copied to clipboard

Nafis Sadeq, Byungkyu Kang, Prarit Lamba, Julian McAuley

DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections (2021.eacl-main)

Challenge:	Masked language modeling (MLM) is often dominated by high-frequency words that are sub-optimal for learning factual knowledge.
Approach:	They propose an approach that forces the model to prioritize informative words in a fully unsupervised way.
Outcome:	The proposed approach significantly improves the performance of pretrained language models on factual recall, question answering, sentiment analysis, and natural language inference in a closed-book setting.

Copied to clipboard

Yury Zemlyanskiy, Sudeep Gandhe, Ruining He, Bhargav Kanagal, Anirudh Ravula, Juraj Gottweis, Fei Sha, Ilya Eckstein

Faithful Question Answering with Monte-Carlo Planning (2023.acl-long)

Challenge:	Using pre-trained models, we learn to jointly predict words and entities from multiple text sources without any human supervision.
Approach:	They propose to learn rich self-supervised entity representations from large amounts of associated text.
Outcome:	The proposed models outperform baseline models on downstream tasks in the TV-Movies domain, and scale to very large corpora.

Copied to clipboard

Ruixin Hong, Hongming Zhang, Hong Zhao, Dong Yu, Changshui Zhang

Multi-style Generative Reading Comprehension (P19-1)

Challenge:	Existing approaches to answer questions using large language models lack the ability to faithfully follow the intermediate reasoning steps from the known premises to the answer.
Approach:	They propose a faithful question-answering task that uses a Monte-Carlo planning algorithm to produce faithful reasoning steps from the known premises to the answer.
Outcome:	The proposed task can produce valid and faithful reasoning steps compared with large language models with a much smaller model size.

Copied to clipboard

Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, Junji Tomita

CodeQA: A Question Answering Dataset for Source Code Comprehension (2021.findings-emnlp)

Challenge:	Current studies on generative reading comprehension (RC) focus on extracting an answer span from textual evidence and natural language generation (NLG).
Approach:	They propose a multi-style abstractive summarization model for question answering called Masque.
Outcome:	The proposed model achieves state-of-the-art performance on the Q&A and Q& A + NLG tasks of MS MARCO and NarrativeQA.

Copied to clipboard

Chenxiao Liu, Xiaojun Wan

Game-theoretic Vocabulary Selection via the Shapley Value and Banzhaf Index (2021.naacl-main)

Challenge:	False. a free-form question answering dataset can serve as a useful research benchmark for source code comprehension.
Approach:	They propose a free-form question answering dataset for source code comprehension . they implement syntactic rules and semantic analysis to transform code comments into question-answer pairs.
Outcome:	The proposed dataset can serve as a useful research benchmark for source code comprehension.

Copied to clipboard

Roma Patel, Marta Garnelo, Ian Gemp, Chris Dyer, Yoram Bachrach

Challenge:	Using the full vocabulary results in less explainable and memory intensive models.
Approach:	They propose a vocabulary selection method that views words as members of a team trying to maximize the model's performance.
Outcome:	The proposed method outperforms baseline models on multiple tasks and datasets.

OKGIT: Open Knowledge Graph Link Prediction with Implicit Types (2021.findings-acl)

Copied to clipboard

. Chandrahas, Partha Talukdar

ERNIE-Doc: A Retrospective Long-Document Modeling Transformer (2021.acl-long)

Challenge:	Open Knowledge Graphs (OpenKGs) are sparse and not directly usable in an end task.
Approach:	They propose a method that bootstraps OpenKGs from a corpus using OpenIE tools.
Outcome:	The proposed method achieves state-of-the-art performance while producing type compatible NPs in the link prediction task.

Copied to clipboard

SiYu Ding, Junyuan Shang, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

KGE-CL: Contrastive Learning of Tensor Decomposition Based Knowledge Graph Embeddings (2022.coling-1)

Challenge:	Existing models for document-level language pretraining are not suitable for long documents due to their quadratically increasing memory and time consumption.
Approach:	They propose a document-level language pretraining model based on Recurrence Transformers.
Outcome:	The proposed model outperforms existing models on language understanding tasks.

Copied to clipboard

Zhiping Luo, Wentao Xu, Weiqing Liu, Jiang Bian, Jian Yin, Tie-Yan Liu

Improving Numerical Reasoning Skills in the Modular Approach for Complex Question Answering on Text (2021.findings-emnlp)

Challenge:	Existing knowledge graph embedding methods ignore semantic similarity between related entities and entity-relation couples in different triples .
Approach:	They propose a contrastive learning framework for tensor decomposition based (TDB) KGE that can shorten the semantic distance of related entities and entity-relation couples in different triples and thus improve the performance of KGE.
Outcome:	The proposed method achieves 51.2% MRR, 46.8% Hits@1 on three standard KGE datasets, 37.8% MRR and 28.6% Hits @1 on FB15k-237 datasets and 59.1% MRR .

Copied to clipboard

Xiao-Yu Guo, Yuan-Fang Li, Gholamreza Haffari

Xiaoyu Bai, Manfred Stede

Challenge:	Neural Module Networks (NMNs) is an end-to-end differentiable model in the programmer-interpreter paradigm.
Approach:	They propose to make the interpreter question-aware and capture the relationship between entities and numbers in both questions and paragraphs.
Outcome:	The proposed models outperform the original models on the DROP dataset and are interpertable by nature.

Copied to clipboard

Challenge:	Recent years have seen increasing interest in applying natural language processing (NLP) applications to the field of education.
Approach:	They propose an NLP-based system that supports german secondary school students in an argumentative writing exercise.
Outcome:	The proposed system will support students in a German school exercise . the system will assess similarity between arguments in snippets of argumentative text .

BERT Shows Garden Path Effects (2023.eacl-main)

Copied to clipboard

Tovah Irwin, Kyra Wilson, Alec Marantz

Few-Shot Question Answering by Pretraining Span Selection (2021.acl-long)

Challenge:	Garden path sentences are sentences that readers incorrectly parse, requiring partial or total re-analysis of the sentence structure.
Approach:	They assess transformer language models which have been fine-tuned on a question-answering task and evaluate their performance on comprehension questions based on garden path and control sentences.
Outcome:	The proposed models have low performance in certain instances of question answering based on garden path contexts, and incorrectly assign semantic roles aligning for the most part with human performance.

Copied to clipboard

Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy

A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC (N19-1)

Challenge:	Pretraining models with recurring span selection are effective, but perform poorly in a few-shot setting.
Approach:	They propose recurring span selection scheme that asks model to select correct span in passage with multiple sets of recurring recurrings.
Outcome:	The proposed model achieves 72.7 F1 on multiple benchmarks while maintaining competitive performance in the high-resource setting.

Copied to clipboard

Mark Yatskar

Few-shot Query-Focused Summarization with Prefix-Merging (2022.emnlp-main)

Challenge:	In response to this development, there have been a flurry of new datasets for question answering.
Approach:	They propose to use SQuAD 2.0, QuAC, and CoQA to provide question answering on textual data.
Outcome:	The proposed datasets provide complementary coverage of the first two aspects, but weak coverage of third.

Copied to clipboard

Ruifeng Yuan, Zili Wang, Ziqiang Cao, Wenjie Li

Summarization

Challenge:	Query-focused summarization has been considered as an important extension for text summarizing . lack of large-scale datasets hinders its development .
Approach:	They propose to integrate text summarization and question answering into a prefix-based pretraining strategy for few-shot learning in query-focused summarizing.
Outcome:	The proposed prefix-based pretraining outperforms fine-tuning on query-focused summarization.

Learning a Cost-Effective Annotation Policy for Question Answering (2020.emnlp-main)

Copied to clipboard

Bernhard Kratzwald, Stefan Feuerriegel, Huan Sun

Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA? (2021.acl-long)

Challenge:	State-of-the-art question answering systems require large amounts of training data for which labeling is time consuming and thus expensive.
Approach:	They propose a framework for annotating QA datasets that entails learning a cost-effective annotation policy and a semi-supervised annotation scheme.
Outcome:	The proposed approach can reduce up to 21.1% of the annotation cost compared with traditional methods . the proposed approach is based on a cost-effective annotation policy and semi-supervised annotation scheme .

Copied to clipboard

Cunxiang Wang, Pai Liu, Yue Zhang

Representations for Question Answering from Documents with Tables and Text (2021.eacl-main)

Challenge:	Existing work is limited in using small benchmarks with high test-train overlaps.
Approach:	They construct a dataset of closed-book QA using SQuAD and investigate the performance of BART.
Outcome:	Experiments show that pre-trained language models can achieve high performance on closed-book QA tasks.

Copied to clipboard

Vicky Zayats, Kristina Toutanova, Mari Ostendorf

EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation (2025.findings-acl)

Challenge:	a study aims to improve question answering on tables by refining table representations based on textual context.
Approach:	They aim to improve question answering from tables by refining table representations based on textual context.
Outcome:	The proposed method improves on the Natural Questions dataset using text and table representations.

Copied to clipboard

Taeho Hwang, Sukmin Cho, Soyeong Jeong, Hoyun Song, SeungYoon Han, Jong C. Park

Lifting the Curse of Multilinguality by Pre-training Modular Transformers (2022.naacl-main)

Challenge:	Current retrieval-augmented generation systems struggle when retrieval models fail to rank the most relevant documents . existing extractive methods reduce latency but rely on independent, non-adaptive sentence selection .
Approach:	They introduce an extractive context compression framework that enhances retrieval-augmented generation in question answering.
Outcome:	EXIT surpasses existing compression methods and uncompressed baselines in QA accuracy . the framework reduces inference time and token count while preserving contextual dependencies .

Copied to clipboard

Jonas Pfeiffer, Naman Goyal, Xi Lin, Xian Li, James Cross, Sebastian Riedel, Mikel Artetxe

Mitigating Bias for Question Answering Models by Tracking Bias Influence (2024.naacl-long)

Challenge:	Recent work on multilingual pre-trained models has focused on pre-training transformers on concatenated corpora of a large number of languages.
Approach:	They propose a language-specific module approach that allows for more languages to be trained post-hoc.
Outcome:	The proposed model can be pre-trained on multiple languages with no drop in performance .

Copied to clipboard

Mingyu Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (D18-1)

Challenge:	Existing literature observes bias in question answering (QA) models, but there is no method to mitigate it.
Approach:	They propose an approach to mitigate the bias of question answering models by observing the influence of a query instance on another instance.
Outcome:	The proposed method reduces bias level in all 9 bias categories while maintaining comparable QA accuracy.

Copied to clipboard

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, Christopher D. Manning

DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling (2020.findings-emnlp)

Challenge:	Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers.
Approach:	They propose a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) the questions provide sentence-level supporting facts required for reasoning; and (4) a type of factoid comparison questions to test QA systems’ ability to extract relevant facts and perform necessary comparison.
Outcome:	The proposed dataset has 113k Wikipedia-based question-answer pairs and four key features that make it challenging for the latest QA systems.

Copied to clipboard

Jiecao Chen, Liu Yang, Karthik Raman, Michael Bendersky, Jung-Jung Yeh, Yun Zhou, Marc Najork, Danyang Cai, Ehsan Emadzadeh

Prompt Tuning for Discriminative Pre-trained Language Models (2022.findings-acl)

Challenge:	Existing knowledge distillation models are not optimized for dealing with pairs (or tuples) of texts.
Approach:	They propose a framework for distilling fast and accurate models on text pair tasks using a scalable end-to-end training strategy.
Outcome:	Empirical studies on academic and real-world e-commerce benchmarks show the proposed framework can achieve speedups of over 350x and minimal quality drop relative to the cross-attention teacher BERT model.

Copied to clipboard

Yuan Yao, Bowen Dong, Ao Zhang, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Leyu Lin, Maosong Sun, Jianyong Wang

Pushing the Limits of AMR Parsing with Self-Learning (2020.findings-emnlp)

Challenge:	Recent studies have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing tasks.
Approach:	They propose a prompt tuning framework that reformulates NLP tasks into a discriminative language modeling problem.
Outcome:	The proposed framework improves on text classification and question answering tasks and prevents unstable tuning problems in low-resource settings.

Copied to clipboard

Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Revanth Gangi Reddy, Radu Florian, Salim Roukos

Open Information Extraction on Scientific Text: An Evaluation (C18-1)

Challenge:	Abstract Meaning Representation (AMR) parsing has experienced a notable growth in performance in the last two years due to the impact of transfer learning and the development of novel architectures specific to AMR.
Approach:	They propose to use AMR annotations to generate synthetic text and refine actions oracle without additional human annotations for AMR parsing.
Outcome:	The proposed models improve on AMR 1.0 and 2.0 without human annotations.

Copied to clipboard

Paul Groth, Mike Lauruhn, Antony Scerri, Ron Daniel Jr.

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)

Challenge:	Open Information Extraction (OIE) is the unsupervised creation of structured information from text.
Approach:	They propose to use two state-of-the-art OIE systems to evaluate the performance of OIE on scientific texts originating from 10 different disciplines.
Outcome:	The proposed methods perform significantly worse on scientific text than encyclopedic text.

Copied to clipboard

Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin

Challenge:	a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities .
Approach:	They present a comparative analysis to identify and distinguish LLM activities from human activities.
Outcome:	The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities.

Improving Health Question Answering with Reliable and Time-Aware Evidence Retrieval (2024.findings-naacl)

Copied to clipboard

Juraj Vladika, Florian Matthes

Large-Scale Relation Learning for Question Answering over Knowledge Bases with Pre-trained Language Models (2021.emnlp-main)

Challenge:	Existing question answering systems rely on pre-selected and annotated evidence documents, thus making them inadequate for addressing novel questions.
Approach:	They propose to use the common retrieve-then-read QA pipeline and PubMed as a trustworthy collection of medical research documents to answer health questions from three diverse datasets.
Outcome:	The proposed approach improves the macro F1 score by 10% by utilizing the common retrieve-then-read QA pipeline and PubMed as a trustworthy collection of medical research documents.

Copied to clipboard

Yuanmeng Yan, Rumei Li, Sirui Wang, Hongzhi Zhang, Zan Daoguang, Fuzheng Zhang, Wei Wu, Weiran Xu

CoReTab: Improving Multimodal Table Understanding with Code-driven Reasoning (2026.eacl-long)

Challenge:	Existing KBQA methods focus on the natural language but ignore textual information carried by the nodes and edges.
Approach:	They propose to perform relation extraction, relation matching, and relation reasoning tasks to align the natural language expressions to the relations in the KB and reason over the missing connections.
Outcome:	Experiments on WebQSP show that the proposed model outperforms baselines even when the KB is incomplete.

Copied to clipboard

Van-Quang Nguyen, Takayuki Okatani

Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models (2024.naacl-long)

Challenge:	Existing datasets for multimodal table understanding provide short factual answers without explicit multi-step reasoning supervision.
Approach:	They propose a code-driven reasoning framework that produces scalable, interpretable, and automatically verifiable annotations by coupling multi-step reasoning with executable Python code.
Outcome:	The proposed model achieves significant gains over baseline models while producing transparent and verifiable reasoning traces.

Copied to clipboard

Seiji Maekawa, Hayate Iso, Sairam Gurajada, Nikita Bhutani

Exploring Diverse Expressions for Paraphrase Generation (D19-1)

Challenge:	Large language models (LMs) excel in retrieving popular facts, but encounter difficulty with infrequent entity-relation pairs compared to retrievers.
Approach:	They propose to use a WiTQA dataset to explore the effects of combinations of entities and relations on LMs.
Outcome:	The proposed model can retain popular relations of less common entities while retaining the same popular relations.

Copied to clipboard

Lihua Qian, Lin Qiu, Weinan Zhang, Xin Jiang, Yong Yu

AMR Beyond the Sentence: the Multi-sentence AMR corpus (C18-1)

Challenge:	Existing neural paraphrase generation methods focus on single paraphrases while ignoring the fact that diversity is essential for enhancing generalization capability and robustness of downstream applications.
Approach:	They propose a novel approach with two discriminators and multiple generators to generate a variety of different paraphrases.
Outcome:	The proposed model gains significant diversity and improves quality over state-of-the-art datasets.

Copied to clipboard

Tim O’Gorman, Michael Regan, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Martha Palmer

Model Surgery: Modulating LLM’s Behavior Via Simple Parameter Editing (2025.naacl-long)

Challenge:	Abstract Meaning Representation (AMR) is limited to capturing the semantics of individual sentences.
Approach:	They propose a corpus that annotates coreference and similar phenomena on top of existing AMRs.
Outcome:	The proposed corpus is compared with existing corpora on sentence-level semantics . it shows that it can be used for information extraction and question answering .

Copied to clipboard

Huanqian Wang, Yang Yue, Rui Lu, Jingxin Shi, Andrew Zhao, Shenzhi Wang, Shiji Song, Gao Huang

Challenge:	Current approaches for detoxification or preventing jailbreaking involve fine-tuning billions of parameters through gradient descent with substantial computational cost.
Approach:	They propose to use supervised fine-tuning and Reinforcement Learning from human feedback to modify LLMs' behavior by directly editing a small subset of parameters.
Outcome:	Experiments show that editing a small subset of parameters can modulate specific behaviors of LLMs, such as detoxification and resistance to jailbreak, with only inference-level computational resources.

Few-Shot Upsampling for Protest Size Detection (2021.findings-acl)

Copied to clipboard

Andrew Halterman, Benjamin J. Radford

The ART of LLM Refinement: Ask, Refine, and Trust (2024.naacl-long)

Challenge:	a common task in social science is "upsampling" coarse document labels to finegrained labels . a new task is proposed for "up-samping" coarse labels to more detailed information .
Approach:	They propose a task for "upsampling" coarse document labels to finegrained labels or spans . they use a question answering format to provide fine-grained label information .
Outcome:	The proposed method outperforms a pre-trained model on a small set of examples but is weaker on fewer examples.

Copied to clipboard

Kumar Shridhar, Koustuv Sinha, Andrew Cohen, Tianlu Wang, Ping Yu, Ramakanth Pasunuru, Mrinmaya Sachan, Jason Weston, Asli Celikyilmaz

Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models (2023.emnlp-main)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations and self-improve?
Approach:	They propose a reasoning with a refinement strategy called ART: Ask, Refine, and Trust that asks necessary questions to decide when an LLM should refine its output and uses it to affirm or deny trust.
Outcome:	The proposed reasoning with a refinement strategy achieves a performance gain of +5 points over baselines on two multistep reasoning tasks.

Copied to clipboard

Kaitlyn Zhou, Dan Jurafsky, Tatsunori Hashimoto

To Answer or Not to Answer (TAONA): A Robust Textual Graph Understanding and Question Answering Approach (2025.findings-emnlp)

Challenge:	a recent study examines how epistemic markers of certainty, uncertainty, or evidentiality affect LMs' attitudes toward knowledge . accuracies of expressions of high certainty and low certainty are associated with questions . prior work focused on learning the mapping between internal probabilities of a model and an ordinal output .
Approach:	They develop a typology of epistemic markers and inject 50 markers into questions . they find that LMs are highly sensitive to epistemical markers in prompts .
Outcome:	The proposed model epistemology study shows that LMs are sensitive to epistemic markers in prompts . expressions of high certainty result in 7% decrease in accuracy, while factive verbs hurt performance .

Copied to clipboard

Yuchen Yan, Aakash Kolekar, Sahika Genc, Wenju Xu, Edward W Huang, Anirudh Srinivasan, Mukesh Jain, Qi He, Hanghang Tong

Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification (2025.emnlp-main)

Challenge:	Existing studies assume that generated answers integrate all relevant information from the textual graph.
Approach:	They propose a novel GraphRAG model that integrates all relevant information from the textual graph into the generated answer.
Outcome:	Extensive experiments validate TAONA’s superior performance for both A-side and B-side tasks.

Copied to clipboard

Jikai Wang, Zhenxu Tian, Juntao Li, Qingrong Xia, Xinyu Duan, Zhefeng Wang, Baoxing Huai, Min Zhang

Challenge:	Existing methods to accelerate autoregressive generation of large language models require training costs.
Approach:	They propose a training-free alignment-augmented speculative decoding algorithm . it leverages the output distribution obtained in the prefilling phase to provide more aligned draft candidates .
Outcome:	The proposed method increases the average generation score by 3.3 points for the LLaMA3 model.

End-to-End Entity Resolution and Question Answering Using Differentiable Knowledge Graphs (2021.emnlp-main)

Copied to clipboard

Amir Saffari, Armin Oliya, Priyanka Sen, Tom Ayoola

Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering (2025.emnlp-main)

Challenge:	End-to-end (E2E) trained models for question answering over knowledge graphs (KGQA) are effective, but training a weakly supervised dataset is difficult.
Approach:	They extend the boundaries of E2E learning for KGQA to include the training of an ER component.
Outcome:	The proposed model is fully differentiable thanks to a recent method for building differentiably KGs.

Copied to clipboard

Hwan Chang, Yumin Kim, Yonghyun Jun, Hwanhee Lee

Cluster-Former: Clustering-based Sparse Transformer for Question Answering (2021.findings-acl)

Challenge:	Large Language Models (LLMs) are increasingly deployed in sensitive domains . large-scale benchmarks for contextual security preservation against attacks remain lacking .
Approach:	They evaluate 10 Large Language Models on a benchmark dataset to assess their adherence to contextual non-disclosure policies.
Outcome:	The proposed model fails to adhere to user-defined security policies in question answering . the model fails in indirect attacks, especially when it violates user-definable policies .

Copied to clipboard

Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Yuwei Fang, Siqi Sun, Yu Cheng, Jingjing Liu

Text Clustering

Challenge:	Existing models for encoding long sequences in deep learning suffer from high latency and memory demands.
Approach:	They propose a clustering-based sparse Transformer framework to perform attention across chunked sequences.
Outcome:	The proposed framework achieves state-of-the-art on several major QA benchmarks.

Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation (2024.emnlp-main)

Copied to clipboard

Jirui Qi, Gabriele Sarti, Raquel Fernández, Arianna Bisazza

Syntax-augmented Multilingual BERT for Cross-lingual Transfer (2021.acl-long)

Challenge:	Recent research has shown that self-citing large language models (LLMs) fail to faithfully reflect their context usage throughout the generation process.
Approach:	They propose a plug-and-play approach using model internals for faithful answer attribution in RAG applications that detects context-sensitive answer tokens and pairs them with retrieved documents contributing to their prediction.
Outcome:	The proposed approach achieves citation quality and efficiency comparable to self-citation while allowing for a finer-grained control of attribution parameters.

Copied to clipboard

Wasi Ahmad, Haoran Li, Kai-Wei Chang, Yashar Mehdad

NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset (2021.findings-emnlp)

Challenge:	Existing studies show that pre-trained multilingual text encoders capture language syntax, helping cross-lingual transfer.
Approach:	They provide language syntax and train mBERT to encode universal dependency tree structure.
Outcome:	The proposed model improves cross-lingual transfer on PAWS-X and MLQA benchmarks by 1.4 and 1.6 points on average across all languages.

Copied to clipboard

Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim

Natural Language Deduction through Search over Statement Compositions (2022.findings-emnlp)

Challenge:	Existing question answering datasets lack numerical reasoning and reasoning processes . current research on numerical reasoning focuses on simple calculations .
Approach:	They propose a conversational and bilingual question answering dataset with numerical reasoning with compound mathematical expressions.
Outcome:	The proposed model achieves 55.5 exact match scores while human performance is 89.7.

Copied to clipboard

Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, Greg Durrett

QUITO-X: A New Perspective on Context Compression from the Information Bottleneck Theory (2025.findings-emnlp)

Challenge:	Existing methods focus on the end-to-end discriminative version of this task, but less work has treated the generative version of the task.
Approach:	They propose a system that decomposes the task into separate steps coordinated by a search procedure and produces a tree of intermediate conclusions that faithfully reflects the system’s reasoning process.
Outcome:	The proposed system proves true statements while rejecting false ones on the EntailmentBank dataset with a 17% absolute higher step validity than the end-to-end T5 model.

Copied to clipboard

Yihang Wang, Xu Huang, Bowen Tian, Yueyang Su, Lei Yu, Huaming Liao, Yixing Fan, Jiafeng Guo, Xueqi Cheng

Linguistic Theories

Challenge:	Existing methods for compressing context by removing redundant tokens are inconsistent with the objective of retaining the most important tokens when conditioning on a given query.
Approach:	They propose a method that uses information bottleneck theory to compress context . they propose to remove redundant tokens using metrics such as self-information or perplexity .
Outcome:	The proposed method achieves a 25% increase in compression rate compared to the state-of-the-art .

CoRI: Collective Relation Integration with Data Augmentation for Open Information Extraction (2021.acl-long)

Copied to clipboard

Zhengbao Jiang, Jialong Han, Bunyamin Sisman, Xin Luna Dong

Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging (2022.findings-emnlp)

Challenge:	Existing methods to integrate extracted knowledge from the Web to knowledge graphs (KGs) however, the predictions are made independently, which can be mutually inconsistent.
Approach:	They propose a relation integration model that aligns free-text relations to relations in a target KG . they propose combining two stages to make independent predictions and a collective model that accesses all candidate predictions.
Outcome:	The proposed model outperforms baseline models on two datasets and improves AUC from .677 to .748 and from 1.716 to 1.780.

Copied to clipboard

Peng Lu, Ivan Kobyzev, Mehdi Rezagholizadeh, Ahmad Rashid, Ali Ghodsi, Phillippe Langlais

Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers (2024.acl-long)

Challenge:	Recent studies show that the flatness of the local minimum correlates well with better generalization.
Approach:	They propose to use a method encouraging convergence to a flatter minimum to fine-tune PLMs.
Outcome:	The proposed method outperforms state-of-the-art methods on NLP tasks without extra computation cost.

Copied to clipboard

Gal Yona, Roee Aharoni, Mor Geva

Zero-Shot Cross-Lingual Transfer with Meta Learning (2020.emnlp-main)

Challenge:	Standard question answering evaluation protocols do not take factual errors into account and compare a predicted answer against answers of a single granularity level.
Approach:	They propose a new evaluation setting where a predicted answer is evaluated in terms of accuracy and informativeness against a set of multi-granularity answers.
Outcome:	The proposed evaluation setting compares a predicted answer against a set of multi-granularity answers.

Copied to clipboard

Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, Isabelle Augenstein

Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval (2021.naacl-main)

Challenge:	There are more than 7,000 languages spoken in the world, over 90 of which have more than 10 million native speakers each.
Approach:	They propose to use meta-learning to train a model on multiple languages at the same time . they use standard supervised, zero-shot cross-lingual, and few-shot crosses-lingual settings for different natural language understanding tasks.
Outcome:	The proposed setup improves on the state-of-the-art for a total of 15 languages.

Copied to clipboard

Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, Hal Daumé III

Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition (2020.emnlp-main)

Challenge:	Current methods for complex question answering use structured knowledge and unstructured text.
Approach:	They propose a multi-step retrieval approach that iteratively forms an evidence chain through beam search in dense representations.
Outcome:	The proposed method is competitive to state-of-the-art systems without using semi-structured information.

Copied to clipboard

Yun He, Ziwei Zhu, Yin Zhang, Qin Chen, James Caverlee

On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems (2025.findings-naacl)

Challenge:	Existing methods to augment pre-trained language models with disease knowledge are lacking.
Approach:	They propose a method to augment BERT-like pre-trained language models with disease knowledge.
Outcome:	The proposed method improves on a suite of BERT models over three tasks.

Copied to clipboard

Juraj Vladika, Florian Matthes

KALA: Knowledge-Augmented Language Model Adaptation (2022.naacl-main)

Challenge:	Retrieval-augmented generation (RAG) is an approach to augment large language models (LLMs) despite their impressive performance, LLMs can generate plausible sounding but factually incorrect responses (hallucinations)
Approach:	They propose to use BM25 and semantic search as retrievers to augment large language models by reducing their reliance on static knowledge and improving answer factuality.
Outcome:	The proposed approach improves QA performance on a biomedical task with up to 15 snippets but stagnates or declines beyond that.

Copied to clipboard

Minki Kang, Jinheon Baek, Sung Ju Hwang

Methods for Numeracy-Preserving Word Embeddings (2020.emnlp-main)

Challenge:	Pre-trained language models (PLMs) have proved to be effective on various natural language understanding tasks.
Approach:	They propose a domain adaption framework which modulates the intermediate hidden representations of PLMs with domain knowledge, consisting of entities and their relational facts.
Outcome:	The proposed framework outperforms adaptive pre-training on question answering and named entity recognition tasks on multiple datasets across different domains.

Copied to clipboard

Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika, Lawrence Carin

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences (2024.acl-long)

Challenge:	Word embedding models capture semantic relationships between words but fail to capture numerical properties associated with numbers.
Approach:	They propose a method to assign and learn embeddings for numbers using word embedders.
Outcome:	The proposed model outperforms pre-trained word embedding models across multiple examples of two tasks.

Copied to clipboard

Yuanhe Tian, Ruyi Gan, Yan Song, Jiaxing Zhang, Yongdong Zhang

Soft Language Clustering for Multilingual Model Pre-training (2023.acl-long)

Challenge:	Current large language models (LLMs) are ineffective in learning domain knowledge and aligning with human preference.
Approach:	They propose a benchmark LLM for Chinese medical domain that uses pre-training, supervised fine-tuning and RLHF to train LLMs.
Outcome:	The proposed LLM performs better than existing LLMs in the Chinese medical domain.

Copied to clipboard

Jiali Zeng, Yufan Jiang, Yongjing Yin, Yi Jing, Fandong Meng, Binghuai Lin, Yunbo Cao, Jie Zhou

Aspect-based Sentiment Analysis in Question Answering Forums (2021.findings-emnlp)

Challenge:	Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from the source language or when pre-training data is limited in size.
Approach:	They propose a method that contextually retrieves prompts as flexible guidance for encoding instances conditionally.
Outcome:	The proposed method improves on the XTREME task and also for low-resource languages in unsupervised sentence retrieval.

Copied to clipboard

Wenxuan Zhang, Yang Deng, Xin Li, Lidong Bing, Wai Lam

Sentiment Analysis

Challenge:	Existing studies on aspects-based sentiment analysis focus on a single opinionated sentence.
Approach:	They propose a model to combine aspects and their sentiments for QA forums . they use cross-sentence aspect-opinion interaction modeling to align the aspect mentioned in the question and associated opinion clues in the answer.
Outcome:	The proposed model outperforms baseline models on three real-world datasets.

Compressing Context to Enhance Inference Efficiency of Large Language Models (2023.emnlp-main)

Copied to clipboard

Yucheng Li, Bo Dong, Frank Guerin, Chenghua Lin

Task-adaptive Pre-training of Language Models with Word Embedding Regularization (2021.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated remarkable power and impressive generalisation abilities across various tasks.
Approach:	They propose a method that prunes redundancies in the input context to make the input more compact.
Outcome:	The proposed method reduces memory and inference time while maintaining comparable performance compared to full context.

Copied to clipboard

Kosuke Nishida, Kyosuke Nishida, Sen Yoshida

Learning Event Graph Knowledge for Abductive Reasoning (2021.acl-long)

Challenge:	Pre-trained language models acquire domain-independent knowledge through pre-training with massive textual resources.
Approach:	They propose a task-adaptive pre-training process that makes static embeddings close to the word embedds obtained in the target domain.
Outcome:	The proposed process improves on BioASQ and SQuAD when the pre-training corpora were not dominated by indomain data.

Copied to clipboard

Li Du, Xiao Ding, Ting Liu, Bing Qin

ReadTwice: Reading Very Large Documents with Memories (2021.naacl-main)

Challenge:	Existing models for abductive reasoning based on formal logic lack commonsense knowledge and effective reasoning mechanism.
Approach:	They propose a narrative text-based abductive reasoning task NLI with a latent variable to capture commonsense knowledge from event graph for guiding the abductive reasoning task.
Outcome:	The proposed model outperforms baseline methods on the abductive reasoning task.

Copied to clipboard

Yury Zemlyanskiy, Joshua Ainslie, Michiel de Jong, Philip Pham, Ilya Eckstein, Fei Sha

CERET: Cost-Effective Extrinsic Refinement for Text Generation (2024.naacl-long)

Challenge:	Existing approaches to model long-range dependencies in text are limited to 512 tokens . however, the amount of compute in attention depends quadratically on the number of tokens in an input text passage.
Approach:	They propose a technique that summarises text into a memory table to be used in a second read of the text.
Outcome:	The proposed method outperforms models of comparable size on several question answering datasets and sets a new state of the art on the NarrativeQA task, with questions about entire books.

Copied to clipboard

Jason Cai, Hang Su, Monica Sunkara, Igor Shalyminov, Saab Mansour

Do You Know That Florence Is Packed with Visitors? Evaluating State-of-the-art Models of Speaker Commitment (P19-1)

Challenge:	Large Language Models (LLMs) generate incomplete, biased or misleading outputs in their initial attempts.
Approach:	They propose a method for refining text generation that takes into account semantic stability, entailment and inter-sample uncertainty measures.
Outcome:	The proposed method outperforms self-consistency and self-rerank baselines under various task setups by 1.6% and 3.5% respectively.

Copied to clipboard

Nanjiang Jiang, Marie-Catherine de Marneffe

Challenge:	Existing models for speaker commitment fail to generalize to diverse linguistic constructions, highlighting directions for improvement.
Approach:	They evaluate two state-of-the-art speaker commitment models on the CommitmentBank . they analyze linguistic correlates of model error on a naturalistic dataset .
Outcome:	The proposed models perform well on some classes but fail to generalize to diverse linguistic constructions.

Counterfactual Adversarial Learning with Representation Interpolation (2021.findings-emnlp)

Copied to clipboard

Wei Wang, Boxin Wang, Ning Shi, Jinfeng Li, Bingyu Zhu, Xiangyu Liu, Rong Zhang

Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering (2020.acl-main)

Challenge:	Existing models with statistical bias are prone to memorized correlations . large pre-trained models such as BERT have revolutionized the model development paradigm in natural language processing .
Approach:	They propose a framework to tackle the problem from a causal perspective using a latent space interpolation approach.
Outcome:	Extensive experiments show that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering.

Copied to clipboard

Vikas Yadav, Steven Bethard, Mihai Surdeanu

What Would a Teacher Do? Predicting Future Talk Moves (2021.findings-acl)

Challenge:	Evidence retrieval is a critical stage of question answering (QA) . Several multi-hop QA datasets have been proposed recently .
Approach:	They propose an unsupervised method that uses only GloVe embeddings to soft-align questions with justification sentences and an iterative process that reformulates queries focusing on terms that are not covered by existing justifications.
Outcome:	The proposed method outperforms all previous methods on the evidence selection task on two datasets: MultiRC and QASC.

Copied to clipboard

Ananya Ganesh, Martha Palmer, Katharina Kann

Challenge:	Recent advances in natural language processing (NLP) have the ability to transform how classroom learning takes place.
Approach:	They propose a task that uses the academically productive talk framework to learn strategies that make for the best learning experience.
Outcome:	The proposed task outperforms baselines on academically productive talk (FTMP) and shows that it outperformed human performance on FTMP.

Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs (D19-1)

Copied to clipboard

Angela Fan, Claire Gardent, Chloé Braud, Antoine Bordes

Automatic Correction of Writing Anomalies in Hausa Texts (2026.acl-long)

Challenge:	Current approaches extract portions of web text as input to Sequence-to-Sequence models . a problem is generating relevant knowledge from noisy and redundant input such as webpages .
Approach:	They propose to restructure free text into local knowledge graphs that are linearized into sequences . they propose to encode the graph as a sequence and then linearize it into a structured sequence .
Outcome:	The proposed method compresses web search information and reduces redundancy . it can be used for open-domain question answering and multi-document summarization tasks .

Copied to clipboard

Ahmad Mustapha Wali, Sergiu Nisioi

Challenge:	Hausa texts are often characterized by writing anomalies such as incorrect character substitutions and spacing errors, which hinder natural language processing (NLP) applications.
Approach:	They propose to fine tune transformer-based Hausa-based models to correct writing anomalies by introducing synthetically generated noise to mimic real-world errors.
Outcome:	The proposed model improves Hausa text quality and improves other low-resource languages.

Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data (P19-1)

Copied to clipboard

Moonsu Han, Minki Kang, Hyunwoo Jung, Sung Ju Hwang

Visual Data in NLP

Challenge:	Existing QA methods lack scalability and performance is difficult to solve with document-level contexts.
Approach:	They propose an end-to-end deep network model that sequentially reads the input contexts into an external memory while replacing memories that are less important for answering unseen questions.
Outcome:	The proposed model improves on a synthetic dataset and real-world large-scale textual and video QA datasets.

Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA (2025.emnlp-main)

Copied to clipboard

Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii

DocNLI: A Large-scale Dataset for Document-level Natural Language Inference (2021.findings-acl)

Challenge:	Large Language Models (LLMs) often struggle with question answering due to hallucinated answers.
Approach:	They propose a multilingual QA dataset with evergreen labels that can be used to evaluate and train large language models.
Outcome:	The proposed model performs well on 12 modern LLMs and EG-E5 classifiers.

Copied to clipboard

Wenpeng Yin, Dragomir Radev, Caiming Xiong

Semi-supervised Training Data Generation for Multilingual Question Answering (L18-1)

Challenge:	Existing studies focus on sentence-level inference, which limits its application in downstream NLP problems.
Approach:	They propose to construct a large-scale dataset for document-level NLI that can be used to study NLP problems.
Outcome:	The proposed model performs well on popular sentence-level benchmarks and generalizes well to out-of-domain NLP tasks that rely on inference at document granularity.

Copied to clipboard

Kyungjae Lee, Kyoungho Yoon, Sunghyun Park, Seung-won Hwang

SuperGLEBer: German Language Understanding Evaluation Benchmark (2024.naacl-long)

Challenge:	Existing datasets for question answering (QA) tasks mostly support only English . however, existing resources for these tasks are labor intensive .
Approach:	They propose to combine Korean QA datasets with machine-translated English resources to build seed resources.
Outcome:	The proposed approach leads to 71.50 F1 on Korean QA (comparable to 77.3 F1)

Copied to clipboard

Jan Pfister, Andreas Hotho

In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)

Challenge:	a new set of German-pretrained models are being released, but no established, diverse and systematic evaluation suite is available for them.
Approach:	They assemble a Natural Language Understanding benchmark suite for the German language and evaluate 10 existing German-pretrained models.
Outcome:	The proposed benchmark suite evaluates 10 German-pretrained models on 29 tasks . the results show that encoder models are good choices for most tasks, but not all .

Copied to clipboard

Chengwei Qin, Aston Zhang, Chen Chen, Anirudh Dagar, Wenming Ye

EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering (2020.emnlp-main)

Challenge:	Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific.
Approach:	They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations.
Outcome:	The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks.

Copied to clipboard

Momchil Hardalov, Todor Mihaylov, Dimitrina Zlatkova, Yoan Dinkov, Ivan Koychev, Preslav Nakov

Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)

Challenge:	EXAMS is a benchmark dataset for cross-lingual and multilingual question answering for high school examinations.
Approach:	They propose to use EXAMS to evaluate cross-lingual and multilingual question answering for high school examinations.
Outcome:	The proposed model can be used to explore multilingual reasoning and knowledge transfer methods and pre-trained models in schools in different languages, which was not possible by now.

Copied to clipboard

Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt

Query Refinement Prompts for Closed-Book Long-Form QA (2023.acl-long)

Challenge:	Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions.
Approach:	They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering.
Outcome:	The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models.

Copied to clipboard

Reinald Kim Amplayo, Kellie Webster, Michael Collins, Dipanjan Das, Shashi Narayan

Contrastive Learning for Task-Independent SpeechLLM-Pretraining (2025.findings-acl)

Challenge:	Large language models (LLMs) can answer questions and produce long-form texts, but the latter is difficult to evaluate since they are subjective in nature.
Approach:	They propose query refinement prompts that encourage LLMs to express multifacetedness and generate long-form answers covering multiple facets of the question.
Outcome:	The proposed model outperforms fully finetuned models in the closed-book setting and retrieve-then-generate open-book models.

Copied to clipboard

Maike Züfle, Jan Niehues

Hierarchical Transformer for Task Oriented Dialog Systems (2021.naacl-main)

Challenge:	Large language models excel in speech processing tasks but their reliance on written text limits their application in real-world scenarios.
Approach:	They propose a task-independent speech pretraining stage and task-specific fine-tuning stage to adapt LLMs to speech processing tasks.
Outcome:	The proposed model outperforms models specialized on speech translation and question answering while being trained on 10% of the task-specific data.

Copied to clipboard

Bishal Santra, Potnuru Anusha, Pawan Goyal

MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data (2022.acl-long)

Challenge:	Existing models for dialog generation are challenging to train using the standard Seq2Seq models.
Approach:	They propose a framework for Hierarchical Transformer Encoders that can be morphed into any hierarchical transformer by using specially designed attention masks and positional encodings.
Outcome:	The proposed framework can be morphed into any hierarchical encoder, including HRED and HIBERT like models, by using specially designed attention masks and positional encodings.

Copied to clipboard

Yilun Zhao, Yunxiang Li, Chenying Li, Rui Zhang

A Nil-Aware Answer Extraction Framework for Question Answering (D18-1)

Challenge:	Existing benchmarks for numerical reasoning over hybrid data only include a single flat table in each document .
Approach:	They propose a new benchmark with QA pairs over multi hierarchical tabular and textual data.
Outcome:	The proposed model is more complex and challenging than existing benchmarks and is available on github . it uses facts retrieving to extract relevant facts from both tables and text and symbolic reasoning over retrieved facts.

Copied to clipboard

Souvik Kundu, Hwee Tou Ng

AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning (2024.emnlp-main)

Challenge:	Recent research suggests that reading comprehension-based question answering systems assume that every question has a valid answer in the associated passage.
Approach:	They propose a novel nil-aware answer span extraction framework that can return Nil or a text span from the associated passage as an answer in a single step.
Outcome:	The proposed framework outperforms baseline approaches on a newsQA dataset.

Copied to clipboard

Hao Sun, Jiayi Wu, Hengyi Cai, Xiaochi Wei, Yue Feng, Bo Wang, Shuaiqiang Wang, Yan Zhang, Dawei Yin

Revisiting Large Language Models as Zero-shot Relation Extractors (2023.findings-emnlp)

Challenge:	Recent advances in large language models have been remarkable . users face a choice between using cloud-based LLMs for generation quality or local-based ones for lower computational cost .
Approach:	They propose a new LLM utilization paradigm that facilitates collaborative operation . they evaluate AdaSwitch across 7 benchmarks and compare it to other LLMs .
Outcome:	The proposed model improves performance of local and cloud agents across 7 benchmarks . it achieves competitive results compared to the cloud agent while utilizing less computational overhead.

Copied to clipboard

Guozheng Li, Peng Wang, Wenjun Ke

An empirical analysis of existing systems and datasets toward general simple question answering (2020.coling-main)

Challenge:	Recent studies show that large language models (LLMs) transfer well to new tasks out-of-the-box . relationship extraction (RE) involves a certain degree of labeled or unlabeled data even under zero-shot setting.
Approach:	They propose a simple prompt recursively using LLMs to transform RE inputs to QA format . they propose qq prompting and qt prompting to improve their results .
Outcome:	The proposed method improves on different model sizes, benchmarks and settings.

Copied to clipboard

Namgi Han, Goran Topic, Hiroshi Noji, Hiroya Takamura, Yusuke Miyao

WiCE: Real-World Entailment for Claims in Wikipedia (2023.emnlp-main)

Challenge:	evaluators of simple factoid question answering using different datasets are not able to solve SimpleQuestions.
Approach:	They evaluate the progress of the field toward solving simple factoid questions over a knowledge base.
Outcome:	The proposed model is nearly solved on the most popular dataset, but not on the robustness of existing systems.

Copied to clipboard

Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, Greg Durrett

ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots (2025.naacl-long)

Challenge:	Textual entailment models are increasingly used in fact-checking, presupposition verification in question answering, or summary evaluation.
Approach:	They propose a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia that provides en-tailment judgments over sub-sentence units of the claim and a minimal subset of evidence sentences that support each subclaim.
Outcome:	The proposed dataset improves on multiple datasets at test time and shows that real claims involve verification and retrieval problems that existing models fail to address.

Copied to clipboard

Yu-Chung Hsiao, Fedir Zubach, Gilles Baechler, Srinivas Sunkara, Victor Carbune, Jason Lin, Maria Wang, Yun Zhu, Jindong Chen

SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples (2023.emnlp-main)

Challenge:	Existing screen datasets focus on low-level structural and component understanding or on a much higher-level composite task such as navigation and task completion for autonomous agents.
Approach:	They propose to annotate 86k question-answer pairs over the RICO dataset to benchmark screen content understanding.
Outcome:	The proposed dataset covers full answers, short answer phrases, and corresponding UI contents with bounding boxes, enabling four subtasks to address various application scenarios.

Copied to clipboard

Deqing Fu, Ameya Godbole, Robin Jia

Challenge:	Manually collecting challenging negative examples can be costly and domain-specific.
Approach:	They propose a method that synthesizes negative examples from only positive ones using a mask infilling model.
Outcome:	The proposed method closes 69.6% of the performance gap on a dataset where half of the evaluation examples are unanswerable compared to a model trained on SQuAD 2.0.

bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark (2023.acl-long)

Copied to clipboard

Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Veselin Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev

A Lightweight Method to Generate Unanswerable Questions in English (2023.findings-emnlp)

Challenge:	bgGLUE is a benchmark for evaluating language models on natural language understanding (NLU) tasks in Bulgarian.
Approach:	They propose to use a benchmark to evaluate language models on NLU tasks in Bulgarian.
Outcome:	The proposed model performs well on sequence labeling tasks, but there is room for improvement for tasks that require more complex reasoning.

Copied to clipboard

Vagrant Gautam, Miaoran Zhang, Dietrich Klakow

FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models (2021.emnlp-main)

Challenge:	Existing approaches to build robust question answering models are too complex . antonym and entity swaps on answerable questions are used to build models .
Approach:	They propose a method for performing antonym and entity swaps on unanswerable questions.
Outcome:	The proposed method outperforms the previous state-of-the-art and has higher human-judged relatedness and readability.

Copied to clipboard

Rakesh Chada, Pradeep Natarajan

Perhaps PTLMs Should Go to School – A Task to Assess Open Book and Closed Book QA (2021.emnlp-main)

Challenge:	Existing pre-trained models need fine-tuning on tens of thousands of examples to achieve good results.
Approach:	They propose a framework that leverages pre-trained text-to-text models and aligns them with their pre-training framework.
Outcome:	The proposed framework outperforms the XLM-Roberta-large on multiple QA benchmarks and is applicable to multilingual situations.

Copied to clipboard

Manuel Ciosici, Joe Cecil, Dong-Ho Lee, Alex Hedges, Marjorie Freedman, Ralph Weischedel

Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation (2020.emnlp-main)

Challenge:	Taking the exam closed book, but having read the textbook, yields at best minor improvement (56%), suggesting that the PTLM may not have “understood” the textbook (or perhaps misundersttoo the questions).
Approach:	They propose to use pre-trained language models to answer questions from introductory college textbooks and hundreds of true/false statements based on review questions written by the authors.
Outcome:	The proposed task includes two college-level introductory texts in the social sciences (American Government 2e) and humanities (U.S. History).

Copied to clipboard

Minki Kang, Moonsu Han, Sung Ju Hwang

Efficient and Accurate Contextual Re-Ranking for Knowledge Graph Question Answering (2024.lrec-main)

Challenge:	Existing methods to train language models on diverse text corpora have brought up performance improvements on several natural language understanding (NLU) tasks.
Approach:	They propose a method to automatically generate domain- and task-adaptive maskings of a given text for self-supervised pre-training.
Outcome:	The proposed framework outperforms rule-based masking strategies on question answering and text classification datasets on which it outperformed rule-driven masking techniques.

Copied to clipboard

Kexuan Sun, Nicolaas Paul Jedema, Karishma Sharma, Ruben Janssen, Jay Pujara, Pedro Szekely, Alessandro Moschitti

On the Importance of Diversity in Question Generation for QA (2020.acl-main)

Challenge:	Existing approaches to QA over textual data are based on a "retrieve-then-generate" pipeline.
Approach:	They propose a "triple-level" labeling strategy that infers fine-grained labels and trains a re-ranker to improve relevance of retrieved triples.
Outcome:	The proposed pipeline improves on prior KGQA systems by 5.56% Exact Match.

Copied to clipboard

Md Arafat Sultan, Shubham Chandel, Ramón Fernandez Astudillo, Vittorio Castelli

Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring (2026.acl-long)

Challenge:	In this paper, we show that textual diversity in automatic question generation is beneficial for downstream QA.
Approach:	They propose to use textual diversity to promote automatic question generation as a quality measure for QA.
Outcome:	The proposed measure of QG quality correlates well with evaluation on QA.

Copied to clipboard

Jamshid Mozafari, Bhawna Piryani, Adam Jatowt

QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)

Challenge:	Existing approaches to estimate question difficulty rely on readability formulas, retrieval-based signals, or popularity statistics.
Approach:	They propose a method that estimates question difficulty by computing the entropy of plausibility scores over candidate answers.
Outcome:	The proposed method outperforms baselines across four QA datasets and shows strong robustness across hyperparameter variations and question types.

Copied to clipboard

Rami Aly, Marek Strong, Andreas Vlachos

HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering (2026.findings-acl)

Challenge:	Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators.
Approach:	They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models.
Outcome:	The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points.

Copied to clipboard

Yuyu Liu, Sarang Rajendra Patil, Mengjia Xu, Tengfei Ma

DTCRS: Dynamic Tree Construction for Recursive Summarization (2025.acl-long)

Challenge:	Recent studies have shown that LLM-based EHR question answering is costly to deploy and does not leverage hierarchical structure of clinical data.
Approach:	They propose a Lorentzian model that embeds codes, visits, and questions in hyperbolic space and answers queries via geometry-consistent cross-attention with type-specific pointer heads.
Outcome:	The proposed model embeds codes, visits, and questions in hyperbolic space and answers queries via geometry-consistent cross-attention with type-specific pointer heads.

Copied to clipboard

Guanran Luo, Zhongquan Jian, Wentao Qiu, Meihong Wang, Qingqiang Wu

Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling (D19-1)

Challenge:	Recursive summarization (RAG) is an important method for mitigating large model hallucinations and enhancing answer interpretability.
Approach:	They propose a method that dynamically generates summary trees based on document structure and query semantics.
Outcome:	The proposed method significantly reduces summary tree construction time and achieves substantial improvements across three QA tasks.

Copied to clipboard

Jinfeng Rao, Linqing Liu, Yi Tay, Wei Yang, Peng Shi, Jimmy Lin

Unlocking Memorization in Large Language Models with Dynamic Soft Prompting (2024.emnlp-main)

Challenge:	Existing techniques for relevance and semantic matching cannot be easily adapted to the other.
Approach:	They propose a model that incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness.
Outcome:	The proposed model incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness.

Copied to clipboard

Zhepeng Wang, Runxue Bao, Yawen Wu, Jackson Taylor, Cao Xiao, Feng Zheng, Weiwen Jiang, Shangqian Gao, Yanfu Zhang

StablePrompt : Automatic Prompt Tuning using Reinforcement Learning for Large Language Model (2024.emnlp-main)

Challenge:	Pretrained large language models excel in a variety of natural language processing tasks . however, they pose significant security risks due to their tendency to memorize training data .
Approach:	They propose a method to estimate LLM memorization using dynamic, prefix-dependent soft prompts.
Outcome:	The proposed method can achieve maximum relative improvement of 135.3% and 39.8% over baseline compared to state-of-the-art methods.

Copied to clipboard

Minchan Kwon, Gaeun Kim, Jongsuk Kim, Haeil Lee, Junmo Kim

NLKI: A Lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks (2025.findings-emnlp)

Challenge:	Recent advances in large language models have made it difficult to find appropriate prompts for tasks with multiple input-output formats.
Approach:	They propose a prompt tuning method based on reinforcement learning (RL) they propose an anchor model and an extension for generating input-dependent prompts.
Outcome:	The proposed method outperforms existing methods on a variety of tasks and achieves State-of-the-art performance across diverse types and sizes of LLMs.

Copied to clipboard

Aritra Dutta, Swapnanil Mukherjee, Deepanway Ghosal, Somak Aditya

DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking (2024.findings-emnlp)

Challenge:	Small vision-language models lag behind their larger generative counterparts due to lack of knowledge.
Approach:	They propose a framework that integrates commonsense knowledge into small vision-language models . the framework retrieves natural language facts and prompts an LLM to craft natural language explanations .
Outcome:	The proposed framework retrieves natural language facts and prompts an LLM to craft natural language explanations.

Copied to clipboard

Devrim Çavuşoğlu, Seçil Şen, Ulaş Sert

CREPE: Open-Domain Question Answering with False Presuppositions (2023.acl-long)

Challenge:	Multiple-choice cloze tests are a prevalent form of assessment that evaluates students' comprehension and inference abilities.
Approach:	They propose a framework for distractor generation using readily available pre-trained language models . human evaluations confirm that their approach produces more effective distractors .
Outcome:	The proposed framework outperforms existing methods without training or fine-tuning human evaluations confirm it.

Copied to clipboard

Xinyan Yu, Sewon Min, Luke Zettlemoyer, Hannaneh Hajishirzi

Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs (2025.findings-emnlp)

Challenge:	Existing question answering datasets assume all questions have well defined answers.
Approach:	They propose a QA dataset containing a distribution of false presuppositions . they find that 25% of questions contain false presumptions .
Outcome:	The proposed model finds that 25% of questions contain false presuppositions . the model can find presuffpositions moderately well, but struggle when predicting correctness .

Copied to clipboard

Essa Jan, Moiz Ali, Muhammad Saram Hassan, Muhammad Fareed Zaffar, Yasir Zaki

Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)

Challenge:	a study shows that comprehension-intensive fine-tuning tasks retain knowledge longer . however, all models exhibit significant performance drops when applying injected knowledge in broader contexts .
Approach:	study: comprehension-intensive fine-tuning tasks achieve higher knowledge retention rates . larger models show improved retention across all task types, study finds .
Outcome:	a new study shows that comprehension-intensive fine-tuning tasks retain knowledge better than mapping-oriented tasks despite exposure to identical factual content.

Copied to clipboard

Xiaoqing Zheng, Jiehang Zeng, Yi Zhou, Cho-Jui Hsieh, Minhao Cheng, Xuanjing Huang

tagE: Enabling an Embodied Agent to Understand Human Instructions (2023.findings-emnlp)

Challenge:	Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension.
Approach:	They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts .
Outcome:	The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials .

Copied to clipboard

Chayan Sarkar, Avik Mitra, Pradip Pramanick, Tapas Nayak

Challenge:	Existing systems for natural language understanding (NLU) are limited due to the inherent ambiguity and incompleteness inherent in natural language.
Approach:	They propose a system to extract tasks from natural language instructions and map them to robots' established collection of skills.
Outcome:	The proposed system outperforms baseline models in the training and evaluation of a dataset featuring complex instructions.

A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot (2023.emnlp-main)

Copied to clipboard

Aanisha Bhattacharyya, Yaman K Singla, Balaji Krishnamurthy, Rajiv Ratn Shah, Changyou Chen

Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs (2024.acl-long)

Challenge:	Existing annotated training datasets hinder development of supervised learning models for multimedia content . lack of annotating benchmarks hinders development of models with satisfactory performance . a recent study shows that large language models have zero-shot performance in multimedia understanding .
Approach:	They propose to verbalize long videos to generate their descriptions in natural language . they then perform video-understanding tasks on the generated story as opposed to the original video .
Outcome:	The proposed method achieves better results than baselines for video understanding.

Copied to clipboard

Tianqing Fang, Zeming Chen, Yangqiu Song, Antoine Bosselut

Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking (2022.emnlp-main)

Challenge:	Currently, language models struggle to generate commonsense inferences for complex tasks due to data scarcity and the difficulty of reasoning over multiple pieces of information.
Approach:	They propose a dataset to generate commonsense inferences from commonsensible data . they use a commonsence knowledge graph to extract and form questions from existing commonseense knowledge graphs.
Outcome:	The proposed dataset improves the ability of language models to reason about complex events without expensive human annotations.

Copied to clipboard

Tim Baumgärtner, Leonardo F. R. Ribeiro, Nils Reimers, Iryna Gurevych

MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer (2020.emnlp-main)

Challenge:	Extensive experiments demonstrate that integrating relevance feedback directly into neural re-ranking models improves their performance.
Approach:	They propose a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant.
Outcome:	The proposed approach outperforms all other methods by 5.2% on large-scale information retrieval datasets.

Copied to clipboard

Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder

Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering (2021.emnlp-main)

Challenge:	Current deep pretrained models lack capacity to represent all languages . limited capacity is an issue even for high-resource languages where models are not included in training data at all.
Approach:	They propose an adapter-based framework that enables high portability and parameter-efficient transfer to arbitrary tasks and languages by learning modular language and task representations.
Outcome:	The proposed framework outperforms state-of-the-art models on cross-lingual transfer across languages and typologically diverse models.

Copied to clipboard

Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpektor, Omri Abend

CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space (2025.emnlp-main)

Challenge:	Existing evaluation methods for factual consistency in knowledge-grounded dialogues are unreliable and limit their applicability.
Approach:	They propose an automatic evaluation metric for factual consistency in knowledge-grounded dialogue using automatic question generation and question answering.
Outcome:	The proposed evaluation metric consistently shows higher correlation with human judgements.

Copied to clipboard

Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, Jincai Huang

Chat or Learn: a Data-Driven Robust Question-Answering System (2020.lrec-1)

Challenge:	Embodied Question Answering (EQA) tasks are primarily focused on indoor environments, leaving the complexities of urban settings unexplored.
Approach:	They propose a task where an embodied agent answers open-vocabulary questions in dynamic city spaces.
Outcome:	The proposed agent achieves 60.7% of human-level answering accuracy compared to baselines . the proposed agent outperforms existing agents in open-ended city spaces .

Copied to clipboard

Gabriel Luthier, Andrei Popescu-Belis

Cross-Policy Compliance Detection via Question Answering (2021.emnlp-main)

Challenge:	QA systems tend to perform poorly at chitchat, while data-driven chatbots are typically user-friendly but not goal-oriented .
Approach:	They propose to use a controller to perform dialogue act classification and feed user input either to a sequence-to-sequence chatbot or to QA systems.
Outcome:	The proposed system is a spoken QA application for the Google Home smart speaker.

Copied to clipboard

Marzieh Saeidi, Majid Yazdani, Andreas Vlachos

Evaluation of Dataset Selection for Pre-Training and Fine-Tuning Transformer Language Models for Clinical Question Answering (2020.lrec-1)

Challenge:	Policy compliance detection is the task of ensuring that a scenario conforms to a policy.
Approach:	They propose to decompose policy compliance detection into question answering . they propose to use an existing dataset to augment expert annotations .
Outcome:	The proposed approach improves accuracy in cross-policy setups, especially when policies are unseen in training.

Copied to clipboard

Sarvesh Soni, Kirk Roberts

SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents (2024.acl-long)

Challenge:	Existing work on EHR QA models that learn to answer questions from structured data has focused on analyzing questions or mapping questions to existing NLP based information extraction models.
Approach:	They conduct 48 experiments on two clinical question answering datasets . they use open-domain and domain-specific corpora to fine-tune Transformer language models .
Outcome:	The proposed models can learn to answer questions from unstructured notes with accuracies up to 90% on open-domain and domain-specific corpora.

Copied to clipboard

Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Yonatan Bisk, Graham Neubig, Hao Zhu

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation (2021.emnlp-main)

Challenge:	Existing studies on building language agents have not addressed this social learning gap.
Approach:	They propose an interactive learning method that improves the social intelligence of language agents by using behavior cloning and self-reinforcement based training on filtered social interaction data.
Outcome:	The proposed method allows a 7B LLM to reach the social goal completion ability of an expert model (GPT-4-based agent) without the loss of more generic abilities, such as the ability to answer knowledge-based questions.

Copied to clipboard

Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Khodra, Ayu Purwarianti, Pascale Fung

Beyond Accuracy Optimization: Computer Vision Losses for Large Language Model Fine-Tuning (2024.findings-emnlp)

Challenge:	Lack of publicly available NLG benchmarks for low-resource languages poses a challenge . authors show that IndoBART and IndoGPT achieve competitive performance on all tasks .
Approach:	They propose a benchmark to measure natural language generation progress in three low-resource languages of Indonesia . they use a corpus of pretraining datasets to build their models .
Outcome:	The proposed benchmark measures progress in Indonesian, Javanese, and Sundanese . the results highlight the importance of pretraining on closely related, localized languages .

Copied to clipboard

Daniele Rege Cambrin, Giuseppe Gallipoli, Irene Benedetto, Luca Cagliero, Paolo Garza

Hallucination Detection in LLMs with Topological Divergence on Attention Graphs (2026.acl-long)

Challenge:	Large Language Models (LLMs) have demonstrated impressive performance across various tasks, but current training approaches combine cross-entropy loss with extensive data, human feedback, or ad hoc methods to enhance performance.
Approach:	They evaluate the use of established semantic segmentation loss functions in natural language generation to create a versatile, practical, and scalable solution for fine-tuning different architectures.
Outcome:	The proposed models improve on Math Word Problems and question answering tasks without additional data or human feedback.

Copied to clipboard

Alexandra Bazarova, Andrei Volodichev, Aleksandr Yugay, Andrey Shulga, Alina Ermilova, Konstantin Polev, Julia Belikova, Rauf Parchiev, Dmitry Simakov, Maxim Savchenko, Andrey Savchenko, Serguei Barannikov, Alexey Zaytsev

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers (2020.emnlp-main)

Challenge:	Large language models (LLMs) are prone to producing so-called hallucinations, i.e., content that is factually or contextually incorrect.
Approach:	They propose a TOpology-based HAllucination detector which quantifies the structural properties of graphs induced by attention matrices.
Outcome:	The proposed detector achieves state-of-the-art or competitive results on several benchmarks while requiring minimal annotated data and computational resources.

Copied to clipboard

Jaemin Cho, Jiasen Lu, Dustin Schwenk, Hannaneh Hajishirzi, Aniruddha Kembhavi

Extrinsic Evaluation of Machine Translation Metrics (2023.acl-long)

Challenge:	Recent work has adapted vision-and-language models to generative tasks like image captioning.
Approach:	They propose an extension to LXMERT with training refinements to generate images from text.
Outcome:	The proposed model can generate images from pieces of text while still being comparable to existing models.

Copied to clipboard

Nikita Moghe, Tom Sherborne, Mark Steedman, Alexandra Birch

Do Question Answering Modeling Improvements Hold Across Benchmarks? (2023.acl-long)

Challenge:	MT metrics are widely used to distinguish the quality of machine translation systems across relatively large test sets.
Approach:	They evaluate the segment-level performance of the most widely used MT metrics by correlating them with how useful they are for downstream tasks.
Outcome:	The MT metrics are widely used to distinguish the quality of machine translation systems across relatively large test sets.

Copied to clipboard

Nelson F. Liu, Tony Lee, Robin Jia, Percy Liang

The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)

Challenge:	a new study finds that human-constructed and downsampled benchmarks hold more concurrence than downsampled benchmarks.
Approach:	They propose to measure concurrence between two QA benchmarks on a set of 20 models . they find that human-constructed benchmarks have high concurrence amongst themselves .
Outcome:	The proposed models hold broadly across the diverse landscape of question answering (QA) benchmarks.

Copied to clipboard

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Qi Cao, Dawei Yin, Huawei Shen, Xueqi Cheng

Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in LLMs (2024.acl-long)

Challenge:	despite near-perfect results, effectiveness of model editing in real-world applications remains unclear.
Approach:	They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework .
Outcome:	The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported.

Copied to clipboard

Bilgehan Sel, Priya Shanmugasundaram, Mohammad Kachuee, Kun Zhou, Ruoxi Jia, Ming Jin

SKRAG: A Retrieval-Augmented Generation Framework Guided by Reasoning Skeletons over Knowledge Graphs (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in tasks such as summarization, arithmetic reasoning, and question answering.
Approach:	They propose a framework that explores decisions’ consequences from multiple stakeholder perspectives and a SKIG framework to enhance moral reasoning in large language models.
Outcome:	The proposed framework exhibits marked improvements compared to baselines across different language models and benchmarks.

Copied to clipboard

Xiaotong Xu, Yizhao Wang, Yunfei Liu, Shengyang Li

DisastQA: A Comprehensive Benchmark for Evaluating Question Answering in Disaster Management (2026.findings-acl)

Challenge:	Existing KG-based question answering frameworks face inefficient subgraph retrieval, limited reasoning capabilities, and high computational costs.
Approach:	They propose a Skeleton-guided RAG framework for knowledge graph question answering . SKRAG leverages a lightweight language model enhanced with the Finite State Machine constraint .
Outcome:	The proposed framework outperforms baselines and general-domain benchmarks on a KGQA dataset in the space science and utilization domain.

Copied to clipboard

Zhitong Chen, Kai Yin, Xiangjue Dong, Chengkai Liu, Xiangpeng Li, Bo Li, Junwei Ma, Yiming Xiao, Ali Mostafavi, James Caverlee

M3Retrieve: Benchmarking Multimodal Retrieval for Medicine (2025.emnlp-main)

Challenge:	Existing benchmarks for question answering (QA) are lacking in a high-stakes environment.
Approach:	They propose a rigorously verified benchmark of 3,000 expert-annotated questions . they propose 'keypoint-based evaluation protocol' emphasizing factual completeness over verbosity .
Outcome:	Experiments with 20 models reveal substantial divergences from general-purpose models such as MMLU-Pro.

Copied to clipboard

Arkadeep Acharya, Akash Ghosh, Pradeepika Verma, Kitsuchart Pasupa, Sriparna Saha, Dr Priti Singh

Question Answering as Programming for Solving Time-Sensitive Questions (2023.emnlp-main)

Challenge:	Strong retrieval models are increasingly important in knowledge-intensive domains.
Approach:	They propose a benchmark to evaluate multimodal retrieval models in medical settings . they examine 1.2 million text documents and 164K multimodal queries .
Outcome:	The proposed model spans 5 domains,16 medical fields, and 4 distinct tasks with over 1.2 Million text documents and 164K multimodal queries.

Copied to clipboard

Xinyu Zhu, Cheng Yang, Bei Chen, Siheng Li, Jian-Guang Lou, Yujiu Yang

mPresenter: An Agentic Framework for Generating Multilingual Presentation Videos from Scientific Papers (2026.findings-acl)

Challenge:	Recent studies show that Large Language Models (LLMs) have shown remarkable intelligence in question answering.
Approach:	They propose to reframe the Question Answering task as Programming to overcome this limitation by leveraging LLMs' superior ability in understanding both natural language and programming language.
Outcome:	The proposed approach improves on time-sensitive question answering datasets by 14.5% over baselines.

Copied to clipboard

Wenhan Han, Xiao Xiao, Mykola Pechenizkiy, Meng Fang

Visual Data in NLP

Challenge:	Existing Paper2Video systems are monolingual and often rely on single-pass pipelines.
Approach:	They propose a multilingual agentic Paper2Video system that decomposes the task into planning, audience-oriented critique, layout-aware slide generation, and multilingual figure interpretation.
Outcome:	The proposed system improves question-answering accuracy relative to previous systems while maintaining affordable cost and latency.

XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models (2023.emnlp-main)

Copied to clipboard

Davis Liang, Hila Gonen, Yuning Mao, Rui Hou, Naman Goyal, Marjan Ghazvininejad, Luke Zettlemoyer, Madian Khabsa

RefuteBench: Evaluating Refuting Instruction-Following for Large Language Models (2024.findings-acl)

Challenge:	Large multilingual models rely on a single vocabulary shared across 100+ languages . this vocabulary bottleneck limits the representational capabilities of multilingual model XLM-R .
Approach:	They propose a new approach for scaling to large multilingual vocabularies by de-emphasizing token sharing between languages with little lexical overlap and assigning vocabulary capacity to achieve sufficient coverage for each individual language.
Outcome:	The proposed model outperforms XLM-R on all language tasks and is particularly effective on low-resource tasks.

Copied to clipboard

Jianhao Yan, Yun Luo, Yue Zhang

TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack (2022.emnlp-main)

Challenge:	The application scope of large language models (LLMs) is expanding . however, evaluating whether models can respond to user feedback has not been thoroughly analyzed.
Approach:	They propose a benchmark to assess whether large language models can respond to refuting feedback and adhere to user demands throughout the conversation.
Outcome:	The proposed benchmark covers tasks such as question answering, machine translation, and email writing.

Copied to clipboard

Yu Cao, Dianqi Li, Meng Fang, Tianyi Zhou, Jun Gao, Yibing Zhan, Dacheng Tao

Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks (2024.lrec-main)

Challenge:	Existing adversarial models rely on keyword matching and ignore relevant contextual relations for answer prediction.
Approach:	They propose to use keyword matching to attack model with two biases that rely on a perturbed answer sentence and a distracting answer sentence to misguide model.
Outcome:	The proposed method produces fluent and grammatical adversarial contexts while maintaining gold answers.

Copied to clipboard

Xiao Pu, Mingqi Gao, Xiaojun Wan

What to Pre-Train on? Efficient Intermediate Task Selection (2021.emnlp-main)

Challenge:	a recent study focused on intrinsic evaluation, which assesses the quality of summaries, e.g. coherence, fluency, and informativeness, but it focused on task-based extrinsic evaluation to determine the usefulness of summarizations.
Approach:	They incorporate three downstream tasks to measure the usefulness of summaries . they find that fine-tuned models produce more useful summary across all three tasks .
Outcome:	The proposed model produces more useful summaries across all three tasks compared to zero-shot models . human evaluation provides more reliable performance assessment compared with automatic methods .

Copied to clipboard

Clifton Poth, Jonas Pfeiffer, Andreas Rücklé, Iryna Gurevych

Green & Sustainable NLP

Challenge:	Existing methods for fine-tuning intermediate tasks are inefficient and expensive.
Approach:	They propose to use a set of 42 intermediate and 11 target English classification, multiple choice, question answering, and sequence tagging tasks to identify the best settings for intermediate transfer learning.
Outcome:	The proposed methods achieve an average Regret@3 of 1% across all target tasks.

Pay Attention to Implicit Attribute Values: A Multi-modal Generative Framework for AVE Task (2023.findings-acl)

Copied to clipboard

Yupeng Zhang, Shensi Wang, Peiguang Li, Guanting Dong, Sirui Wang, Yunsen Xian, Zhoujun Li, Hongzhi Zhang

KGConv, a Conversational Corpus Grounded in Wikidata (2024.lrec-main)

Challenge:	Existing approaches to extract attribute values from product descriptions are incomplete and noisy due to the tedious nature of this task.
Approach:	They propose a framework to extract attributes from product descriptions to acquire implicit attributes in addition to the explicit ones.
Outcome:	The proposed framework outperforms existing methods on the extraction of implicit attribute values while achieving comparable performance for the explicit ones.

Copied to clipboard

Quentin Brabant, Lina M. Rojas Barahona, Gwénolé Lecorvé, Claire Gardent

Debarun Bhattacharjya, Balaji Ganesan, Junkyu Lee, Radu Marinescu, Katya Mirylenka, Michael Glass, Xiao Shou

Challenge:	a large corpus of 71k English conversations contains on average 8.6 questions . Unlike open domain and task-oriented dialogues, information seeking conversations are driven by the desire to acquire or evaluate knowledge.
Approach:	They propose a large corpus of 71k English conversations where each question-answer pair is grounded in a Wikidata fact.
Outcome:	The proposed dataset can be used for knowledge-based, conversational question generation . it can also be used to generate single-turn questions from Wikidata triples, question rewriting, question answering from conversation or knowledge graphs and quiz generation.

Copied to clipboard

MedREQAL: Examining Medical Knowledge Recall of Large Language Models via Question Answering (2024.findings-acl)

Challenge:	Uncertainty quantification (UQ) provides measures of uncertainty, such as an estimate of the confidence in an LLM’s generated output.
Approach:	They propose a black-box approach where consistency is used as a proxy for confidence in a model's output.
Outcome:	The proposed methods are primarily but not necessarily entirely black- box, with consistency between output and other sampled generations used as a proxy for confidence in its correctness.

Copied to clipboard

Juraj Vladika, Phillip Schneider, Florian Matthes

Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025.acl-long)

Challenge:	Large language models can encode knowledge during pre-training on large text corpora, enabling downstream tasks like question answering (QA).
Approach:	They construct a dataset derived from systematic reviews to examine their ability to encode medical knowledge and their recall.
Outcome:	The proposed model performs well on the biomedical QA dataset.

Copied to clipboard

Wenyu Huang, Pavlos Vougiouklis, Mirella Lapata, Jeff Z. Pan

FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions (2023.emnlp-main)

Challenge:	Multi-hop Question Answering (MHQA) adds layers of complexity to question answering tasks.
Approach:	They explore how LMs respond to multi-hop questions by permuting search results under various configurations.
Outcome:	The proposed model outperforms decoder-only models in MHQA tasks despite being significantly smaller in size .

Copied to clipboard

Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Bras, Gunhee Kim, Yejin Choi, Maarten Sap

Linguistics & Cognitive NLP

Challenge:	Existing evaluations for theory of mind (ToM) use passive narratives that lack interactivity.
Approach:	They propose a benchmark to stress-test ToM within information-asymmetric conversational contexts via question answering.
Outcome:	The proposed benchmark is challenging for state-of-the-art language models, which perform significantly worse than humans even with chain-of thought reasoning or fine-tuning.

If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data (2026.findings-acl)

Copied to clipboard

Yanjun Cui, Ali Emami, Temiloluwa Prioleau, Nikhil Singh

ACCEPT: Adaptive Codebook for Composite and Efficient Prompt Tuning (2024.findings-emnlp)

Challenge:	Current patient platforms only offer static summaries which do not support inquisitive user queries.
Approach:	They propose a framework for question answering over personal glucose data that uses large language models to provide a reasoning engine that selects analytical functions.
Outcome:	The proposed framework achieves 94% value accuracy on synthetic queries and 88% on ambiguous real-world queries.

Copied to clipboard

Yu-Chen Lin, Wei-Hua Li, Jun-cheng Chen, Chu-Song Chen

RARR: Researching and Revising What Language Models Say, Using Language Models (2023.acl-long)

Challenge:	Prompt Tuning has been a popular fine-tuning method for large-scale pretrained language models.
Approach:	They propose a method that allows all soft prompts to share a set of learnable codebook vectors in each subspace, with each prompt differentiated by a number of adaptive weights.
Outcome:	The proposed method achieves superior performance on 17 diverse natural language tasks including natural language understanding (NLU) and question answering (QA) tasks by tuning only 0.3% of parameters of the PLMs.

Copied to clipboard

Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu

FacLens: Transferable Probe for Foreseeing Non-Factuality in Fact-Seeking Question Answering of Large Language Models (2025.emnlp-main)

Challenge:	Language models (LMs) excel at many tasks but often produce unsupported or misleading content.
Approach:	They propose a system that finds attribution for any text generation model and post-edits it to fix unsupported content.
Outcome:	The proposed system improves attribution while preserving the original output.

Copied to clipboard

Yanling Wang, Haoyang Li, Hao Zou, Jing Zhang, Xinlei He, Qi Li, Ke Xu

MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources (2024.findings-acl)

Challenge:	Existing non-factuality detection methods require response generation, which incurs significant computational overhead.
Approach:	They propose a lightweight model called Factuality Lens which effectively probes hidden representations of fact-seeking questions for the NFP task.
Outcome:	The proposed model is able to probe hidden representations of fact-seeking questions and reduce development costs.

Copied to clipboard

Dongkyu Lee, Chandana Satya Prakash, Jack FitzGerald, Jens Lehmann

E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning (2025.emnlp-main)

Challenge:	Existing retrieval-augmented models typically retrieve information from a single type of knowledge source.
Approach:	They propose an efficient memory-augmented transformer to retrieve relevant knowledge from multiple knowledge sources.
Outcome:	The proposed model outperforms existing retrieval-augmented models on popular QA benchmarks in terms of accuracy and speed.

Copied to clipboard

Zihan Liao, Jun Wang, Hang Yu, Lingxiao Wei, Jianguo Li, Jun Wang, Wei Zhang

Multi-Attribute Steering of Language Models via Targeted Intervention (2025.acl-long)

Challenge:	Considerable efforts have been and are still being put into increasing the context length of Large Language Models (LLMs)
Approach:	They propose an approach that divides long contexts into chunks, compresses each into soft prompts using a pretrained text encoder, and aligns these representations with a decoder-only LLM via an adapter.
Outcome:	The proposed approach outperforms 8 state-of-the-art methods in effectiveness and efficiency for document summarization and question answering, and achieves the best performance on LongBench v2 among models of comparable size.

Copied to clipboard

Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

Comparative Analysis of the Intrinsic Metrics for Tokenizers and their effect on Downstream Tasks for Hindi and Marathi (2026.acl-long)

Challenge:	Existing approaches for steering large language models fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity.
Approach:	They propose a steering framework for selective token-level intervention across multiple attributes that enforcing sparsity and orthogonality among vectors for different attributes.
Outcome:	The proposed framework outperforms existing ITI and parameter-efficient fine-tuning approaches across question answering tasks and generative tasks.

Copied to clipboard

Shagun Dwivedi, Kaushik Gopalan

Don’t Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understanding (2026.findings-acl)

Challenge:	Various studies have shown that the performance of language models is poor in non-English or non-European languages.
Approach:	They propose a grapheme cluster tokenizer which shows better performance than other popular tokenizers.
Outcome:	The proposed tokenizers show better or competitiveness on question-answering tasks . the proposed tokenization model is highly correlated to the performance of other tokenizer models .

Copied to clipboard

Quanwei Tang, Dong Zhang, Shoushan Li, Guodong Zhou

ASTRA: Adaptive Semantic Tree Reasoning Architecture for Complex Table Question Answering (2026.acl-long)

Challenge:	Existing question answering (QA) datasets for long audio meetings suffer from acoustic information loss and poor long-term dependency capture.
Approach:	They propose a question answering dataset that captures three core dimensions of long-form audio meeting content.
Outcome:	The proposed model captures three core dimensions of long-form audio meeting content: complex semantics, multi-speaker interactions, and quite long timestamps.

Copied to clipboard

Xiaoke Guo, Songze Li, Zhiqiang Liu, Zhaoyan Gong, Yuanxiang Liu, Huajun Chen, Wen Zhang

Differentiated Vision: Unveiling Entity-Specific Visual Modality Requirements for Multimodal Knowledge Graph (2025.findings-emnlp)

Challenge:	Existing serialization methods fail to capture explicit hierarchies and lack schema flexibility . Existing tree-based approaches suffer from limited semantic adaptability .
Approach:	They propose a method that leverages the global semantic awareness of LLMs to reconstruct tables into Logical Semantic Trees.
Outcome:	The proposed method achieves state-of-the-art (SOTA) performance on complex table benchmarks.

Copied to clipboard

Minghang Liu, Yinghan Shen, Zihe Huang, Yuanzhuo Wang, Xuhui Jiang, Huawei Shen

Augment before You Try: Knowledge-Enhanced Table Question Answering via Table Expansion (2025.findings-emnlp)

Challenge:	Existing methods to extract features from images of entities overlook varying relevance of visual information across entities.
Approach:	a new model integrates structural and multimodal information of entities into a multimodal knowledge graph . a model evaluates the necessity of visual modality for each entity based on its attributes .
Outcome:	The proposed model improves on existing methods by adjusting visual data to different entity types.

Copied to clipboard

Yujian Liu, Jiabao Ji, Tong Yu, Ryan A. Rossi, Sungchul Kim, Handong Zhao, Ritwik Sinha, Yang Zhang, Shiyu Chang

MED-COREASONER: Reducing Language Disparities in Medical Reasoning via Language-Informed Co-Reasoning (2026.acl-long)

Challenge:	Existing methods to integrate external information into a given table neglect the structured nature of the table.
Approach:	They propose a simple yet effective method to integrate external information into a given table by first building an augmenting table and then generating a SQL query over the two tables to answer the question.
Outcome:	The proposed method outperforms strong baselines on three table QA benchmarks.

Copied to clipboard

Fan Gao, Sherry T. Tong, Jiwoong Sohn, Jiahao Huang, Junfeng Jiang, Ding Xia, Piyalitt Ittichaiwong, Kanyakorn Veerakanjana, Hyunjae Kim, Qingyu Chen, Edison Marrese-Taylor, Kazuma Kobayashi, Akiko Aizawa, Irene Li

Memory-Efficient Fine-Tuning of Transformers via Token Selection (2024.emnlp-main)

Challenge:	Existing models that use English and local languages have a multilingual gap . a language-informed co-reasoning framework can be used to improve multilingual reasoning .
Approach:	They propose a language-informed co-reasoning framework that elicits parallel English and local-language reasoning and abstracts them into structured concepts.
Outcome:	Experiments show that Med-CoReasoner improves multilingual reasoning performance by 5% . the framework produces clinically sound and culturally grounded reasoning traces .

Copied to clipboard

Antoine Simoulin, Namyong Park, Xiaoyi Liu, Grey Yang

ReTAG: Retrieval-Enhanced, Topic-Augmented Graph-Based Global Sensemaking (2025.findings-emnlp)

Challenge:	Existing methods for fine-tuning require caching of intermediate activations to update weights during the backward pass.
Approach:	They develop a method to reduce memory usage in fine-tuning of transformers by backpropagating through just a subset of input tokens.
Outcome:	The proposed method reduces memory usage and memory footprint on large transformer models . it can be easily combined with existing methods like LoRA, reducing memory cost .

Copied to clipboard

Boyoung Kim, Dosung Lee, Sumin An, Jinseong Jeong, Paul Hongsuck Seo

Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models (2025.acl-long)

Challenge:	a prior graph-based approach to global sensemaking lacks retrieval mechanisms, topic specificity, and incurs high inference costs.
Approach:	They propose a RetrievalEnhanced, Topic-Augmented Graph framework that retrieves relevant summaries from a topic.
Outcome:	The proposed framework improves response quality while significantly reducing inference time compared to the baseline.

Copied to clipboard

James Flemings, Bo Jiang, Wanrong Zhang, Zafar Takhirov, Murali Annavaram

Ethical NLP

Challenge:	Prior work has shown that privacy leakage of parametric knowledge often occurs from memorized pre-training data.
Approach:	They propose a metric that builds on differential privacy to estimate the privacy leakage of contextual knowledge during decoding by comparing parametric and contextual knowledge.
Outcome:	The proposed method overestimates the privacy leakage of parametric knowledge while separating parametric and contextual knowledge.

HyperKGR: Knowledge Graph Reasoning in Hyperbolic Space with Graph Neural Network Encoding Symbolic Path (2025.emnlp-main)

Copied to clipboard

Lihui Liu

Exploration-Driven Reinforcement Learning for Expert Routing Improvement in Mixture-of-Experts Language Models (2025.findings-emnlp)

Challenge:	Existing methods for linking knowledge graphs are incomplete and rely on Euclidean embeddings . a hyperbolic GNN framework embeds recursive learning trees in hyperbolical space .
Approach:	They propose a hyperbolic GNN framework that embeds recursive learning trees in hyperbolical space and generates query-specific embeddings.
Outcome:	The proposed framework outperforms state-of-the-art methods on multiple benchmark datasets.

Copied to clipboard

Gyunyeop Kim, Sangwoo Kang

CSTree-SRI: Introspection-Driven Cognitive Semantic Tree for Multi-Turn Question Answering over Extra-Long Contexts (2025.acl-long)

Challenge:	MoE-based LLMs are not explicitly supervised to select suitable experts.
Approach:	They propose Exploration-Driven Reinforcement Learning (ERL) which explicitly optimizes the router by exploration of alternative routing paths.
Outcome:	The proposed method improves summarization (SAMSum, XSUM, question answering, and language modeling), and raises routing quality, delivering 8.9 higher MRR than baselines over 100 perturbed routing paths.

Copied to clipboard

Zhaowen Wang, Xiang Wei, Kangshao Du, Yiting Zhang, Libo Qin, Yingjie Xia, Li Kuang

Sightation Counts: Leveraging Sighted User Feedback in Building a BLV-aligned Dataset of Diagram Descriptions (2025.acl-long)

Challenge:	Large Language Models (LLMs) have achieved remarkable success in natural language processing (NLP), particularly in single-turn question answering (QA) on short-text.
Approach:	They propose a framework that captures logical correlations across chunks of ELC and maintains coherence of multi-turn Questions.
Outcome:	The proposed framework is able to capture logical correlations across chunks of ELC and maintain coherence of multi-turn Questions.

Copied to clipboard

Wan Ju Kang, Eunki Kim, Na Min An, Sangryul Kim, Haemin Choi, Ki Hoon Kwak, James Thorne

Visual Data in NLP

Challenge:	Existing studies show that direct generation of diagram descriptions is costly and biased against blind and low-vision (BLV) users.
Approach:	They ask sighted individuals to assess diagram descriptions generated by vision-language models . they use latent supervision to guide the models with latent inference .
Outcome:	The results show that visual descriptions generated by vision-language models are effective and useful to educators who are themselves BLV and teach visually impaired learners.

Learning to Reason Over Time: Timeline Self-Reflection for Improved Temporal Reasoning in Language Models (2025.acl-long)

Copied to clipboard

Adrián Bazaga, Rexhina Blloshmi, Bill Byrne, Adrià de Gispert

Quantifying Uncertainty in Natural Language Explanations of Large Language Models for Question Answering (2025.findings-emnlp)

Challenge:	Large Language Models struggle with temporal reasoning, which requires processing time-related information such as event sequencing, durations, and inter-temporal relationships.
Approach:	They propose a framework that enhances the temporal reasoning abilities of Large Language Models (LLMs) by combining timeline construction with iterative self-reflection.
Outcome:	The proposed framework improves the temporal reasoning abilities of large language models and improves traceability of the inference process.

Copied to clipboard

Yangyi Li, Mengdi Huai

Open-DeBias: Toward Mitigating Open-Set Bias in Language Models (2025.findings-emnlp)

Challenge:	Large language models (LLMs) have shown strong capabilities, enabling concise, context-aware answers in question answering tasks.
Approach:	They propose a framework that provides valid uncertainty guarantees for LLMs . they also propose 'model-agnostic' uncertainty estimation method that maintains valid guarantees even under noise.
Outcome:	The proposed method provides valid uncertainty guarantees even under noise.

Copied to clipboard

Arti Rani, Shweta Singh, Nihar Ranjan Sahoo, Gaurav Kumar Nayak

Training Language Model to Critique for Better Refinement (2025.findings-acl)

Challenge:	Existing approaches to addressing harmful biases in LLMs are limited to predefined categories . a novel, data-efficient, and parameter-efficient debiasing method is proposed to mitigate existing social and stereotypical biase .
Approach:	They propose an open-set bias detection and mitigation method to address harmful biases in text-based QA.
Outcome:	The proposed method improves QA accuracy on Korean BBQ dataset by nearly 48% on ambiguous subsets and 6% on disambiguated ones.

Copied to clipboard

Tianshu Yu, Chao Xiang, Mingchuan Yang, Pei Ke, Bosi Wen, Cunxiang Wang, Jiale Cheng, Li Zhang, Xinyu Mu, Chuxiong Sun, Minlie Huang

TALON: A Multi-Agent Framework for Long-Table Exploration and Question Answering (2025.emnlp-main)

Challenge:	Large language models (LLMs) have remarkable evaluation and critique capabilities, providing insightful feedback and identifying flaws in various tasks.
Approach:	They propose a framework to train critic models using refinement signals to generate feedback loops where critiques guide the model in refining its responses.
Outcome:	The proposed framework outperforms traditional methods and open-source models in terms of critique quality and refinement outcomes.

Copied to clipboard

Ruochun Jin, Xiyue Wang, Dong Wang, Haoqi Zheng, Yunpeng Qi, Silin Yang, Meng Zhang

Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models (2025.acl-long)

Challenge:	Existing approaches to query-relevant content retrieval fail to retrieve contextually relevant data.
Approach:	They propose a multi-agent framework for table question answering over long tables . TALON features a planning agent that iteratively invokes a tool agent to access tabular data .
Outcome:	The proposed framework achieves average accuracy improvements of 7.5% and 12.0% across all language models.

Copied to clipboard

Ido Cohen, Daniela Gottesman, Mor Geva, Raja Giryes

How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucination (2025.emnlp-main)

Challenge:	Vision-language models excel at extracting and reasoning about information from images, yet their capacity to leverage internal knowledge about specific entities remains underexplored.
Approach:	They propose a dataset which allows separating entity recognition and question answering . they hypothesize that this decline arises from limitations in how information flows from image tokens to query tokens.
Outcome:	The proposed model performance drops when the entity is presented visually rather than textually.

Copied to clipboard

Saad Obaid Ul Islam, Anne Lauscher, Goran Glavaš

RePanda: Pandas-powered Tabular Verification and Reasoning (2025.acl-long)

Challenge:	despite LLMs becoming increasingly multilingual, most studies on detecting and quantifying LLM hallucination are English-centric .
Approach:	They train a multilingual hallucination detection model and conduct a large-scale study across 30 languages and 6 open-source LLM families.
Outcome:	The proposed model is based on an English-centric model and annotates gold data for five high-resource languages.

Copied to clipboard

Atoosa Chegini, Keivan Rezaei, Hamid Eghbalzadeh, Soheil Feizi

PICTURE: Enhancing Theory-of-Mind in Large Language Models by Revealing, Not Hiding, Characters’ Lack of Knowledge (2026.acl-long)

Challenge:	Existing methods for fact-checking tabular data rely on black-box models with opaque reasoning.
Approach:	They propose a structured fact verification approach that translates claims into executable pandas queries.
Outcome:	The proposed method outperforms existing methods and demonstrates strong OOD robustness.

Copied to clipboard

Eojin Jeon, SangKeun Lee

OG-RAG: Ontology-grounded retrieval-augmented generation for large language models (2025.emnlp-main)

Challenge:	Existing approaches to simulating Theory of Mind (ToM) using LLMs have been a longstanding problem in natural language processing (NLP).
Approach:	They propose a new method that enables LLMs to generate a character’s lack of knowledge within free-form Chain-of-Thought (CoT) based on this method, they propose to generate perspective-taking outputs as free- form explanations without event hiding.
Outcome:	The proposed method outperforms existing prompting methods by an average of 7.3% on false-belief tasks.

Copied to clipboard

Kartik Sharma, Peeyush Kumar, Yunqing Li

Calibrating Model-Based Evaluation Metrics for Summarization (2026.findings-acl)

Challenge:	Existing retrieval-augmented models fail to account for structured domain knowledge, leading to suboptimal context generation.
Approach:	They propose an Ontology-Grounded Retrieval Augmented Generation method to enhance LLM-generated responses by anchoring retrieval processes in domain-specific ontologies.
Outcome:	The proposed method increases recall of accurate facts by 55% and improves response correctness by 40% across four different LLMs.

Copied to clipboard

Hongye Liu, Dhanajit Brahma, Ricardo Henao

Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts (2026.findings-acl)

Challenge:	Recent advances in summary evaluation are based on model-based metrics to assess quality dimensions, such as completeness, conciseness, and faithfulness.
Approach:	They propose a general framework that generates individual and average proxy scores without relying on reference summaries, human annotations, or expensive model-based metrics.
Outcome:	The proposed framework outperforms baselines on seven datasets on continuous-value scenarios, such as summarization, but is applicable to discrete-value tasks, such QA.

Copied to clipboard

Azher Ahmed Efat, Seok Hwan Song, Wallapak Tavanapong

PROBE: PROcess-Based BEnchmark for Hallucination Detection (2026.findings-acl)

Challenge:	Existing research on chart understanding has been limited to single chart images.
Approach:	They propose a dataset specifically designed for question answering over multi-chart images.
Outcome:	The proposed method shows a 27.4% LLM-based accuracy drop on human-authored questions and a 5.39% gain in the human-generated questions.

Copied to clipboard

Yu Zhang, Peter Belcak, Shizhe Diao, Yonggan Fu, Shaona Ghosh, Morteza Mardani, Eileen Margaret Peters Long, Bei Yu, Pavlo Molchanov